Research paper in NDSS 2010: Improving Spam Blacklisting Through Dynamic Thresholding and Speculative Aggregation

Our blacklist paper titled:

Improving Spam Blacklisting Through Dynamic Thresholding and Speculative Aggregation

and authored by:

Sushant Sinha, Michael Bailey, and Farnam Jahanian
University of Michigan, Ann Arbor, MI - 48109.

is going to be presented in Network and Distributed System Security (NDSS) Symposium, 2010 from 28th Febuary to 3rd March in San Deigo, California.

Here is the abstract:

Abstract:

Unsolicited bulk e-mail (UBE) or spam constitutes a significant
fraction of all e-mail connection attempts and routinely frustrates
users, consumes resources, and serves as an infection vector for
malicious software. In an effort to scalably and effectively reduce
the impact of these e-mails, e-mail system designers have increasingly
turned to blacklisting. Blacklisting (blackholing, block listing) is a
form of course-grained, reputation-based, dynamic policy enforcement
in which real-time feeds of spam sending hosts are sent to networks so
that the e-mail from these hosts may be rejected. Unfortunately,
current spam blacklist services are highly inaccurate and exhibit
both false positives and significant false negatives. In this paper, we
explore the root causes of blacklist inaccuracy and show that the
trend toward stealthier spam exacerbates the existing tension between
false positives and false negatives when assigning spamming IP
reputation. We argue that to relieve this tension, global aggregation
and reputation assignment should be replaced with local aggregation
and reputation assignment, utilizing preexisting global spam
collection, with the addition of local usage, policy, and reachability
information. We propose two specific techniques based on this premise,
\emph{dynamic thresholding} and \emph{speculative aggregation}, whose
goal is to improve the accuracy of blacklist generation. We
evaluate the performance and accuracy of these solutions in the
context of our own deployment consisting of 2.5 million production
e-mails and 14 million e-mails from spamtraps deployed in 11 domains
over a month-long period. We show that the proposed approaches
significantly improve the false positive and false negative rates when
compared to existing approaches.

Anonymization of Court Judgments

Indian Kanoon has been pretty useful for people seeking information on Indian Law. Over time Indian Kanoon reputation on general search engines like google and yahoo improved and as a result pages from Indian Kanoon started surfacing in the top few of search results. So one completely inadvertent thing that happened was that people would go on google/yahoo and type someone's name and see the indiankanoon court judgment related to that person. As many of these judgments are on issue that may be societally bad, many people started feeling embarrassed by their court judgments. To be fair in many cases these people were right. But definitely one has to read the judgments to realize that.

The next thing that happened was that many folks sent me legal threats that I should remove court judgments corresponding to them. Some even claimed that they are currently citizen of countries in which such information is private and I should remove these documents immediately. A few nuanced requests only asked for restricting search engines from indexing these documents by putting these URLs under robots.txt.

As court judgments constitute public records, removing them from the website was out of question. However, restricting generic search engines from indexing such judgments was not that bad. One problem is that someone may be looking for information on a person and he or she will miss that because it is not indexed by generic search engines. After all a person acquitted by a court judgment is only legally right and societal or individual values may differ. So people should be free to decide on the issue by reading these court judgments. The second issue is that these court judgments also brings new users to Indian Kanoon and make them more knowledgable about law. So till now I have been doing nothing with such requests.

Recently a law suit has been filed in Andhra Pradesh High Court seeking for name anonymization in one of the court judgments. The order is here. That particular court judgment refers to a copyright infringement case related to a woman. It seems strange that court will ask to anonymize the name in this case as it does not relate to a woman being a victim of a sexual nature and not a minor. However, the AP high court has taken up this writ petition and it seems possible that the court may decide in the favor of plaintiff.

One question that is of interest is what is the bar for name anonymization in court judgments. It seems like if there is no such bar, then no one would want their name on the records. My search on Indian Kanoon did not reveal much as it seems like this issue has not been addressed in depth till now.


The search on IK reveals a bunch of cases when names have been withheld: http://indiankanoon.org/search/?formInput="name+withheld"

However, almost all of these cases are when a woman is a victim of sexual nature. So it will be interesting to know if the courts order anonymization in cases related to copyright infringement.

MCC Murgugappa Gold Cup 2009 - Schedule and Results

MCC Murugappa Gold Cup Hockey tournament 2009 - October 1 to October 11, 2009 in Mayor Radhakrishanan Stadium, Chennai Egmore

Pool A
------
Air India
Indian Overseas Bank (IOB)
Karnataka XI
BPCL
ONGC

Pool B
-------
Indian Oil Corporation (IOC)
Army XI
Mumbai XI
Indian Railways
Punjab National Bank (PNB)

01-10-2009	Thursday	2:15 pm	IOB vs Karnataka XI	2-2
02-10-2009	Friday	2:15 pm	Railways vs Army XI	1-2
		4:00 pm	Air India vs ONGC	1-5
		5:45 pm	IOC vs Mumbai X1	6-3
03-10-2009	Saturday	2:15 pm	Karnataka XI vs ONGC	1-3
		4:00 pm	Mumbai XI vs Railways	0-2
		5:45 pm	Army XI vs PNB	3-1
04-10-2009	Sunday	2:15 pm	Karnataka XI vs BPCL	1-3
		4:00 pm	Army XI vs IOC	0-1
		5:45 pm	Air India vs IOB	2-0
05-10-2009	Monday	2:15 pm	IOC vs PNB	4-1
		4:00 pm	IOB vs ONGC	1-3
		5:45 pm	Air India vs BPCL	0-3
06-10-2009	Tuesday	4:00 pm	Mumbai XI vs PNB	1-4
		5:45 pm	IOC vs Railways	1-0
07-10-2009	Wednesday	2:15 pm	IOB vs BPCL	0-2
		4:00 pm	Air India vs Karnataka XI	3-3
		5:45 pm	Army XI vs Mumbai XI	3-1
08-10-2009	Thursday	4:00 pm	BPCL vs ONGC	3-4
		5:45 pm	Railways vs PNB	3-2
09-10-2009	Friday		REST DAY
10-10-2009	Saturday	TBA	Winner A vs Runner B>/td>
		TBA	Runner A vs Winner B
11-10-2009	Sunday	TBA	Finals