Are AI article summarizers reliable? We tested a few

This article was first published on August 28, 2019 on the LIZHI.SHOP blog (lizhi.io, now offline), and is being republished here with some adjustments.

A few days ago I wrote "How Long Has It Been Since You Sat Down with a Book?" — and it's understandable that when a busy daily grind eats into leisure time, many people find it hard to put their mind into reading.

So whether in China or abroad, fast-food reading has become the mainstream of our time. From Twitter to Weibo, feeds limit how long a post can be, and short videos are growing more popular than text around the world — and some developers couldn't sit still.

"No one wants to read long articles anymore? Then let me save everyone some time."

"I Lazy To Read"

I Lazy To Read is a web-based tool that, with its blunt name, took the top spot on ProductHunt for a day.

No matter how long the text on a website is, you just copy and paste the URL, and it summarizes it into just 5 sentences.

Many users overseas say it's very handy when browsing forums and reading posts; but when it comes to dense, hard-to-follow articles, how well do these article summarizers actually work?

Testing a long CNN article

I then picked a long article from CNN's homepage explaining "how neutron star collisions produce gold."

But the 5 sentences I Lazy To Read pulled out didn't even hit the keywords in the article's title...

This site doesn't look all that reliable, so let's try a more advanced tool.

An advanced version of I Lazy To Read

Two months after I Lazy To Read appeared, the virtually identical TLDR This also climbed to the top of ProductHunt for a day in July (just how badly do users overseas need tools like this?)

It's also a web-based tool (with a Chrome extension too), and it works the same way: copy and paste a URL or a passage, and it summarizes it into 5 sentences.

This time I decided to test it with a Wikipedia page (you can probably tell from the link, so no need to show the page~):

What was the result? Very curious —

Are you satisfied with these 5 sentences summing up a U.S. president's biography? If you're also looking for a long-text summarizer, why not give it a try yourself and then decide whether it fits your reading habits.

Does AI make it better?

I then came across Gamayun, a site much like the two tools above, though its developer claims to use AI technology.

This time I tried an article on 9to5mac about an iOS update:

Same steps — after copying and pasting the URL, the result was much better than before. Instead of seemingly picking 5 random, unimportant sentences, it condensed the piece into a fairly coherent short text, which is probably the effect users want most.

Are you waiting for a Chinese-language tool?

So far no mature text summarization tool seems to have caught on in China. A developer on GitHub, letiantian, offers a similar open-source algorithm — TextRank.

It can extract keywords and summaries (key sentences) from a text, and splits the text into 4 formats depending on what you need:

In other words, for a long article you can get either a set of keywords or a set of key sentences. If you're interested, you can learn more~

If you also keep an eye on verifiable productivity tools like these, you might also want to check out the selection of genuine software curated at the LIZHI.SHOP software store.

Finally, some of you may be wondering: can the long passages in a book be condensed too?

Here we can use CamScanner, a tool that supports iOS / Android devices — point it at the book, scan, and the text becomes an electronic document you can copy~

Paste the text straight into the tools mentioned above, and it's like summarizing a physical book!

The finished tools introduced in this article were mostly developed outside China, so most of them only support English. If similar Chinese-language tools with reliable enough results appear in China in the future, would you use them? Feel free to leave your thoughts in the comments below~