Or why burglars need to improve their language skills before applying for a job in the army
You are probably not aware of it, but serendipity plays an important role in search, or at least in the type of search we call ‘discovery’. In this blogpost we share a case in which the Alexandria.Works Enterprise Search Engine discovered a remarkable relationship between 2 texts. 2 Related documents that came to light with the help of our Topical Facets™ a proprietary feature of the search engine. A bit of what happens in our brains when we dream. Establishing meaningful relationships between flashing parts.
The Merriam-Webster Dictionary defines serendipity as “The faculty or phenomenon of finding valuable or agreeable things not sought for; also: an instance of this“. And Dictionary.com formulates it as “An aptitude for making desirable discoveries by accident“.
Serendipity is the accidental finding of something valuable or pleasant. It is an accidental discovery. And that’s what this blogpost is all about. To those fun, unexpected discoveries of related documents.
A machine with a dream
During our research into the Natural Language Processing (NLP) technology for Alexandria.Works, the focus was on developing unsupervised information discovery in unstructured text without any human intervention. Stimulated by the sentence “What if a machine could go out and gather information and then ask its own questions” from the MIT Technology Review, we wondered what would happen if our system were to generate its own questions.
Well, usually not much. We let the program autogenerate random strings for a while and presented them again as a question. Without much result. This means that no semantic relationships were found in the data. Until the Alexandria.Works Engine linked the following two texts together:
Text 1 from the New York Times (NYT19981001.0277):
No doubt John Boorman’s canny, elegant new film “The General”, about the notorious Irish thief Martin Cahill, hits unusually close to home. (…) Cahill’s biographer, Paul Williams, maintains that when Cahill tried to enlist in the Navy at 15, in 1964, and filled out an application: “Martin chose the position of bugler”. Unfortunately, due to his difficulties in school, he misread the word as “burglar”.
Text 2, also from the New York Times (NYT19981003.0145 & NYT19981003.0220):
(…) “We’re placing a high value on learning another language”, said Jay Doolan, the director of standards and professional development with the New Jersey Department of Education. “We feel it’s not only appropriate for competing in a shrinking world, but that learning another language also helps you do better in your first language.”
Both articles are from the New York Times, the first appeared on October 1, 1998, the second two days later. The texts were not linked by tags or taxonomies, no training was applied. And yet we can only conclude from this that burglars need to improve their language skills before applying to the army.
A bit like what happens in our brains when we dream. Dreams that, according to neuroscience, are random associations of neuronal activity. And because our brain is actually a pattern recognition machine, a dream is a meeting of meaningful relationships between different parts that flash around while we sleep.
We are not saying that all searches should be serendipitous. But when you are searching in ‘discovery mode’, serendipity can be a great help. And not many search engines offer you such a feature.
Could you use some more serendipity when you are exploring your company’s vast set of documents? Are you curious to find out what might emerge from your organization’s collective memory? Then don’t hesitate and request a test now. It’s free. And it’s the best way to experience the difference.