Social network Twitter has opened up its whole catalogue of messages to a search engine, meaning that its entire content, around 500 billion documents, is available to sift through online.
Twitter launched around eight and a half years ago, and now claims to have some 284 million monthly active users.
"Since that first simple tweet over eight years ago, hundreds of billions of tweets have captured everyday human experiences and major historical events," he wrote.
"Our search engine excelled at surfacing breaking news and events in real time, and our search index infrastructure reflected this strong emphasis on 'recency'.
"But our long-standing goal has been to let people search through every tweet ever published. We [have] built a search service that efficiently indexes roughly half a trillion documents and serves queries with an average latency of under 100ms."
The search engine has the entire canon of 140-character messages at its disposal, and Zhuang suggested that it would have a range of applications, particularly for companies or individuals looking for content on an incident or event.
The new index is 100 times larger than the previous one, and grows by "several billion tweets a week", according to the engineer. This could be a good time for Twitter users to go back into their accounts and delete any dubious tweets to prevent them from resurfacing and embarrassing them.
The system is a mix of batched data aggregation, partitioning and indexing, and is scalable, easy to use and dependable, according to Zhuang.
"Our fixed-size real-time index clusters are non-trivial to expand; adding capacity requires re-partitioning and significant operational overhead. We needed a system that expands in place gracefully."
just setting up my twttr— Jack (@jack) March 21, 2006
Cotton seedling freezes to death as Chang'e-4 shuts down for the Moon's 14-day lunar night
Fortnite easily out-earns PUBG, Assassin's Creed Odyssey and Red Dead Redemption 2 in 2018
Meteor showers as a service will be visible for about 100 kilometres in all directions
Saturn's rings only formed in the past 100 million years, suggests analysis of Cassini space probe data
New findings contradict conventional belief that Saturn's rings were formed along with the planet about 4.5 billion years ago