Farm Development A portable record player for the Internet

I'm very pleased to announce a website I just launched:

I'm calling it "a portable record player for the Internet" because I'm a record collector myself and listening to as much music as possible has been the best way to expand my collection. Yep, I'm one of those guys who goes to record conventions and brings a portable record player.

There are a few places to find used vinyl online — since 161,783 records are for sale on eBay right now, I'd say the Internet is a good venue. The problem is that you can't always listen to a record before you buy it and it's impossible to find great music by name alone when there is so much to choose from. I'm hoping to change all that with this site.

As you can see, it's still in very experimental stages and leaves a lot to be desired. eBay is the only marketplace supported so far but I plan to integrate Music Stack next. If anyone knows of another marketplace, please let me know. Also, if you have a friend who buys records then please send on the link (I'm interested to hear feedback from record buyers).

It took me about 6 months to get the data in order (doing all this in my spare time) so it is very exciting to be able to release an actual site now. The data pipeline is home-grown ETL and uses the eBay API + the super speedy and tolerant lxml.html (contrib by Ian Bicking), the excellent SQLAlchemy, and other tools. As I take the data pipeline to the next level I really hope to start using the very cool SnapLogic ETL framework for Python.

On the UI side this has been my first chance to really dive into Dojo. I like Dojo because it takes semantic HTML seriously, namespacing seriously, unit testing seriously, and object oriented JS seriously. It's a bit monolithic and I still haven't figured out how to build a custom minified version but so far I like it. Also, I really wish the Dojo API reference was more comprehensive. On the backend UI side, I'm using Pylons since I like how well it integrates with 3rd party Python libs. I don't like some of the magic it inherited from Rails but that is pretty easy to work around. Oh, I almost forgot to mention SoundManager 2, the ultimate javascript/flash mp3 player.

  • Re: A portable record player for the Internet

    snaplogic looks awesome. We are using SSIS (SQL Server built in ETL tool) which uses the vile and abominable VB for scripting (tho I guess C# plus presumably Iron Python will be available down the road).

    Oh how I'd love to use Python for all my ETL needs but currently I'm it as far as in-house Python ppl...

  • Re: A portable record player for the Internet

    Wow, pretty neat. How does it work?

    1. How do you link the ebay auction to the record info in the database (the "record record")?

    2. How do you link the record to the audio, which seems to be across multiple sites?

    Or is all this metadata coming in through the eBay API? I don't understand. Works well, though!

  • Re: A portable record player for the Internet

    Hi Mike. It's pretty simple so far but as the project evolves I plan to make it a little more complex. Currently, the backend process crawls the ebay API feed of new auctions for patterns in auction titles ("soul 45," etc) to determine if it's a record for sale. The only auctions currently listed are those where the seller has posted an mp3 in the auction description itself. That's it. The audio files are across multiple sites because they are hosted by the seller.

    In the future I plan to match auctions for records that *do not* have links to samples. I have a prototype of this working but it needs refinement. This is trickier and involves pattern matching techniques to link an auction of a record to audio samples from a past auction of that same record. I have over 22,000 samples cached so I am pretty sure that once I can get an acceptable success rate that it will work.

    I'm also working on another strategy that uses artist / album / song data to actually say "this record is by this artist and is of this album." That will also be tricky but I have some ideas for how to approach it. This will probably yield a higher success rate but I haven't prototyped it out yet.

Note: HTML tags will be stripped. Hit enter twice for a new paragraph.

Recent Projects

  • JSTestNet

    Like botnet but for JS tests in CI.

  • Nose Nicedots

    Nose plugin that prints nicer dots.

  • Fudge

    Mock objects for testing.

  • Fixture

    Loading and referencing test data.

  • NoseJS

    Nose plugin that runs JavaScript tests for a Python project.

  • Wikir

    converts reST to various Wiki formats.