A recent upgrade to the Indian Kanoon server broke down the glibc dependency of many software packages. Glibc is one of the most critical pieces of libraries whose upgrade has definitely hoaxed the box. I came to the data center today trying to fix this problem and even the package manager was not working.
Booting from scratch and then reinstalling core components has produced some results. At least the package manager is working now and I am able to recompile glibc. The compilation is till going on and after these packages that depend on glibc need to be recompiled. Hopefully the server should be up soon.
I am extremely sorry for my mistake due to which other folks are suffering. And I promise not to do blind updates on a production server from now on.
http://www.pluggd.in/indian-startups/indiankanoon-search-engine-for-indian-laws-and-court-judgements-1035/#comments
UPDATE-1
Unfortunately the server could not be fixed today and had to bring the server to home. The server is fixed now but not accessible at indiankanoon.org You can access the server at http://141.212.108.248. You can get the search and access to documents. However, forums are not back yet because of the domain verification recaptcha.net. The server will be put back in the data center tomorrow and it will be back on indiankanoon.org Since the DNS queries are cached for a day it is not worth modifying the DNS entry.
UPDATE-2
The server is resored in the data center as of 1:00 AM IST.
Tuesday, July 28, 2009
Friday, July 24, 2009
Final PhD Defense: Context Aware Network Security
Computer Science and Engineering
CSE Defense
Context Aware Network Security
Sushant Sinha
Friday, July 31, 2009
3:00 pm - 5:00 pm
3725 CSE
Chair: F. Jahanian
The public is invited to attend
CSE Defense
Context Aware Network Security
Sushant Sinha
Friday, July 31, 2009
3:00 pm - 5:00 pm
3725 CSE
Chair: F. Jahanian
The public is invited to attend
Thursday, May 21, 2009
New Text Search Goodies in Postgresql 8.4
Postgresql has a full text search engine built into it. Teodor Siagev and Oleg Bartunov, who started a text search engine called OpenFTS, merged their code base into Postgresql 7.4 as a separate contrib module called tsearch2.
Tsearch2 is highly extensible and sophisticated code base for text search. It is flexible in the sense that you can write your own stemmer and parser, or even use the default one for any new language. Tsearch2 keeping up with the tradition of extensibility in Postgres provides users to define their own ranking function or headline generation. Besides that it provides people to use two text indexes namely GIST and GIN both with different performance curves for initial indexing and index updates. Tsearch2 was merged in the core of Postgres last year in the 8.3 release.
Postgres 8.4 is being released after more than a year of development and testing. A few patches that many people wanted like a default replication scheme in the core and a SELinux in Postgres called SE-Postgrresql were punted for 8.5. The main reasons being that these patches were too big for people to review in the last months.
Postgres 8.4 brings a number of improvements in text search. Here are the list of new text features that you may look out for and may force you to upgrade:
1. Optimizer selectivity function for @@ text search operations (Jan Urbanski)
This was a Google Summer of Code (GSoC) project taken up by Jan Urbanski and it is great that the project was successful. Though it took a lot of time for this patch to be accepted, it provides a quite accurate selectivity measure for text search. This will enable the Postgres planner to produce better plans for SQL queries when text search matching operator is combined with other equally complicated operators.
2. Fast prefix matching in full text searches (Teodor Sigaev, Oleg Bartunov)
Earlier prefix matching was used to be done using LIKE operator. For example, for searching all documents that have have the beginning few words "sush", the WHERE clause needs to contain LIKE 'sush%'. However, LIKE operator does not use the text index and is very slow. This patch introduces fast prefix matching in Postgres. Now the where clause can be something like:
xt_tsvector @@ to_tsquery('sush:*')
3. Support multi-column GIN indexes (Teodor Sigaev)
Earlier if you have to index two separate text columns like "documents" and "comments", then you could only have separate indexes for each column. And then a query has to be matched with each column (q @ document and q @ comments). With this patch, such queries can take advantage of multi-column indexes if the developer has used one. Here is the performance improvement of multi-column index over single index as observed by Teodor:
Multicolumn index vs. 2 single column indexes
Size: 539 Mb 538 Mb
Speed: *1.885* ms 4.994 ms
Index: ~340 s ~200 s
Insert: 72 s/10000 66 s/10000
4. Improve full text search headline() function to allow extracting several fragments of text (Sushant Sinha)
This patch was contributed back by me. Headline generation is the identification of text fragment in a document where query terms appear. The default headline generation function shows only one text fragment for a set of query terms. Further, the existing headline generation function did not show good headlines (it is a more subjective judgment as there is no way to identify a good fragment.
My patch allowed more than one non-overlapping fragment to be displayed for a set of query terms. Further, the text fragments that were chosen in such a way that those fragments contained query items in the most compact way. A lot of databases will be envious of production quality headline generation in postgres now.
5. Improve support for Nepali language and Devanagari alphabet (Teodor)
I do not know much about this but looking at the CVS log, here was the bug that was fixed:
"Some languages have symbols with zero display's width or/and vowels/signs which
are not an alphabetic character although they are not word-breakers too.
So, treat them as part of word."
Tsearch2 is highly extensible and sophisticated code base for text search. It is flexible in the sense that you can write your own stemmer and parser, or even use the default one for any new language. Tsearch2 keeping up with the tradition of extensibility in Postgres provides users to define their own ranking function or headline generation. Besides that it provides people to use two text indexes namely GIST and GIN both with different performance curves for initial indexing and index updates. Tsearch2 was merged in the core of Postgres last year in the 8.3 release.
Postgres 8.4 is being released after more than a year of development and testing. A few patches that many people wanted like a default replication scheme in the core and a SELinux in Postgres called SE-Postgrresql were punted for 8.5. The main reasons being that these patches were too big for people to review in the last months.
Postgres 8.4 brings a number of improvements in text search. Here are the list of new text features that you may look out for and may force you to upgrade:
1. Optimizer selectivity function for @@ text search operations (Jan Urbanski)
This was a Google Summer of Code (GSoC) project taken up by Jan Urbanski and it is great that the project was successful. Though it took a lot of time for this patch to be accepted, it provides a quite accurate selectivity measure for text search. This will enable the Postgres planner to produce better plans for SQL queries when text search matching operator is combined with other equally complicated operators.
2. Fast prefix matching in full text searches (Teodor Sigaev, Oleg Bartunov)
Earlier prefix matching was used to be done using LIKE operator. For example, for searching all documents that have have the beginning few words "sush", the WHERE clause needs to contain LIKE 'sush%'. However, LIKE operator does not use the text index and is very slow. This patch introduces fast prefix matching in Postgres. Now the where clause can be something like:
xt_tsvector @@ to_tsquery('sush:*')
3. Support multi-column GIN indexes (Teodor Sigaev)
Earlier if you have to index two separate text columns like "documents" and "comments", then you could only have separate indexes for each column. And then a query has to be matched with each column (q @ document and q @ comments). With this patch, such queries can take advantage of multi-column indexes if the developer has used one. Here is the performance improvement of multi-column index over single index as observed by Teodor:
Multicolumn index vs. 2 single column indexes
Size: 539 Mb 538 Mb
Speed: *1.885* ms 4.994 ms
Index: ~340 s ~200 s
Insert: 72 s/10000 66 s/10000
4. Improve full text search headline() function to allow extracting several fragments of text (Sushant Sinha)
This patch was contributed back by me. Headline generation is the identification of text fragment in a document where query terms appear. The default headline generation function shows only one text fragment for a set of query terms. Further, the existing headline generation function did not show good headlines (it is a more subjective judgment as there is no way to identify a good fragment.
My patch allowed more than one non-overlapping fragment to be displayed for a set of query terms. Further, the text fragments that were chosen in such a way that those fragments contained query items in the most compact way. A lot of databases will be envious of production quality headline generation in postgres now.
5. Improve support for Nepali language and Devanagari alphabet (Teodor)
I do not know much about this but looking at the CVS log, here was the bug that was fixed:
"Some languages have symbols with zero display's width or/and vowels/signs which
are not an alphabetic character although they are not word-breakers too.
So, treat them as part of word."
Subscribe to:
Posts (Atom)