As a final project for my Internet Auctions course (COS 444), I decided to examine different bidding patterns across the various categories of Ebay and attempt to glean useful demographic information from the correlations from the comparisons of the bidding profiles for each of these categories.

The first stage of the project- and one that proved relatively tricky- was to develop a data scraper that would download very large sets of information from Ebay for further analysis. Complicating matters, Ebay had decided only recently to begin anonymizing user information. This resulted in a surprising variety of different data available on pages of the same type, where some users had been anonymized and others remained visible.

I developed a web scraper in python that was able to look through all of the categories available on Ebay and collect hundreds of thousands of individual item sales. Further processing on the item sales themselves allowed me to glean when exactly individuals were bidding.

Once I had the bidding profile information, it was relatively simple to simply compare any two categories to see how similar/dissimilar their bidding profiles were. Many of the results were surprising, though it was highly encouraging to see a larger percentage made good sense. For example, there is a high level of correlation between individuals buying cell phones and individuals purchasing other electronic items. There is a similar correlation between individuals interested in collectibles and those interested in art.