Simon Willison: In praise of functional programming
On my reading list...
On my reading list...
I think the main difference - that is often overlooked - here is the fact that the semantic approaches offer one step further. They can find related text that share no keywords at all, but might share neighbours
[btw. the 'bayesian approach' I talk about in my essay is nothing less than a glorified keyword-occurence-comparison...]
[Numerik] Der Strassen-Algorithmus ist einer der schnellen Algorithmen zur Multiplikation von Matrizen. Der Trick ist im Grunde, die Matrix so in kleinere Matrizen zu zerlegen, dass man weniger multipliziert und dafür mehr addiert. Ist nett in der Werkstatt Multiplikation und in diesem Skript Effiziente Algorithmen und Datenstrukturen erklärt. Wer sich weiter mit Matrizenmultiplikation auseinander setzen möchte, dürfte auf dieser Seite gut aufgehoben sein: Fast Parallel Matrix Multiplication - Strategies for Practical Hybrid Algorithms.
This site indexes the contents of eleven weblogs over the period March - June 2002 ( a total of 1440 posts ). The goal of this search engine is to demonstrate a fuzzy-search technique called latent semantic indexing.
LSI has been around for some time, remember?
All this will of course be handled in my essay, once it is finished, if ever
Peer-to-Peer Semantic Search Engines: Building a Memex
via Kellan
Contextual Network Search (CNS) is a graph-based alternative to LSI which NITLE began work on in 2003. In a 1981 dissertation at the University of Illinois, Scott Preece describes an almost identical technique under the name spreading activation search, but we have not been able to find further work in this direction. If you are aware of such work, please let us know.
New module Search::ContextGraph... that's from April 2002, but it doesn't seem to be in CPAN... ah author: Maciej Ceglowski, okay we know that one from his 'building a memex' slides... So where is that Perl code?
correction: the CPAN date is 27 April 2003, so this stuff is to new to be listed...
Once again I'm doing a self-experiment. 'Is the Mozilla Composer suited as a writing tool?'.
[update:] About 2 hours into writing an essay about search approaches I could say that, yes Mozilla Composer can be used as a writing tool. Esp. the 'insert Table of Contents' function is mega usefull. COmposer is not a killer writing environment, but definetly usefull. Esp. if you plan to publish your writing in HTML...
[update:] A very early version of the mentioned essay can be found here
Blozom is a standalone Mozilla based weblogging and note management application. It uses plain text files. Blog entries have associated XML files with post metadata. Blozom will support wiki like markup and a link glossary.
I wish I could get it to run... almost there now.
trust master-of-the-mask erik-with-a-k natzke to come up with the world's coolest flash page turning effect in his latest work for hp. (the page effect is in the dreamworks section. but other stuff in the site is nice too.)
[Thu Apr 24 22:49:50 2003] vs.pl: reading archive from ../html/blog/archive
[Thu Apr 24 22:49:50 2003] vs.pl: found 1257 postings
[Thu Apr 24 22:49:50 2003] vs.pl: setting up VectorSpace
[Thu Apr 24 22:49:50 2003] vs.pl: building Index
[Thu Apr 24 22:49:50 2003] vs.pl: Making word list:
[Thu Apr 24 22:50:01 2003] vs.pl: Finished with word list
[Thu Apr 24 22:50:01 2003] vs.pl: doing queries
[Thu Apr 24 22:50:01 2003] vs.pl: making cosines for doc 0,
2003-04-24_22-33
[Thu Apr 24 22:50:02 2003] vs.pl: writing ../html/blog/archive/sims/2003-04-24_22-33.sim
[Thu Apr 24 22:50:02 2003] vs.pl:
[Thu Apr 24 22:50:02 2003] vs.pl: making cosines for doc 1, 2003-04-24_22-29
[Thu Apr 24 22:50:03 2003] vs.pl: writing ../html/blog/archive/sims/2003-04-24_22-29.sim
[Thu Apr 24 22:50:03 2003] vs.pl:
[Thu Apr 24 22:50:03 2003] vs.pl: making cosines for doc 2, 2003-04-24_22-26
[Thu Apr 24 22:50:05 2003] vs.pl: writing ../html/blog/archive/sims/2003-04-24_22-26.sim
[Thu Apr 24 22:50:05 2003] vs.pl:
[Thu Apr 24 22:50:05 2003] vs.pl: making cosines for doc 3, 2003-04-24_15-53
[Thu Apr 24 22:50:06 2003] vs.pl: writing ../html/blog/archive/sims/2003-04-24_15-53.sim
[Thu Apr 24 22:50:06 2003] vs.pl:
[Thu Apr 24 22:50:06 2003] vs.pl: making cosines for doc 4, 2003-04-24_10-41
so the first optimisation could be to go semi-incremental. No need to re-do all 1200+ entries in each run, as most of them don't really change.
next step surly would be to pre-generate the word-index phase, which could also be used for the keyword search.
each actual 'query' takes roughly a second to preform, that is rather fast if you consider you are comparing one doc to 1200+ others
And no, keeping the whole VectorSpace in-memory between user-queries is not really an option. The Perl VM size for this is about 125MB on my machine, which is ok for running it every so often, but not for running it contiously.
Another gripe is the fact that it's difficult to incrementally add new words (or docs containing new words) to a 'pre-generated' space.
This is almost like a search function, but somewhat better. To search you ned to know what you are looking for, esp. in terms of keywords. VextorSpace similar entries form the other side of that. 'Search/Query by Example' sort of. Now the ultimate would be a combination of both, keyword search and similar entries...
Some ideas/wishes I have for that:
In this software package you find a decoder for nexTView - an Electronic TV Programme Guide for the analog domain (as opposed to the various digital EPGs that come with most digital broadcasts). It enables you to receive and browse free TV programme listings for all of the major networks in Germany, Austria, France and Switzerland.
requires a TVtuner card in the PC
XMLTV is a set of utilities to manage your TV viewing. They work with TV listings stored in the XMLTV format, which is based on XML. The idea is to separate out the backend (getting the listings) from the frontend (displaying them for the user), and to implement useful operations like picking out your favourite programmes as filters that read and write XML documents.
At present there are backends grabbing TV listings for Canada, the USA, the UK, Germany, Austria, Sweden, Norway, Finland, New Zealand, Spain, Italy and the Netherlands.
freshmeat.net: Project details for FreeGuide TV Guide
FreeGuide TV Guide is a TV guide program which grabs listings for many countries from the Internet and allows you to view them and construct a personalised guide to print out. It uses the XMLTV tools to grab listings, and requires Java 1.4 or above.
Nachdem TVGenial jetzt nur noch an zahlende Kunden das ganze Listing liefert...