Topic Detection

One of the sub-tasks in determining the topic is determining who is being talked about in a particular text. I had tried Named Entity Recognition using the Open-NLP toolkit and it looks like it did not identify all the "names" in the text. A simple frequency based approach seems to have worked better. What did i do - well, eliminate all the stopwords there are numerous stopword lists available on the list and thereafter create a frequency distribution table of the remaining words and looked at the really frequent words - words occuring above mean + 5 SD. This gave mostly proper nouns and were indeed about the person being talked about. Therefore, the named-entity recogniton tool too should have detected all these instances of words although i still need to look at the same further.

I tried using my summarization project that i did the last semester but that does not happen to suit this task. Perhaps, i should look at the text and try and look at a distinguishing feature... I am yet to try out the topic toolbox.