Farm Development

Thoughts on The Future

back to all thoughts

Python Package Index (PyPI) Mirrors

posted in Google App Engine, Python, The Future by kumar on Tuesday Jul 27th, 2010 at 1:05p.m.

Most deployment systems depend on PyPI, the Python Package Index, for fetching and installing dependencies. Although performance and reliability has greatly improved, there are still days when you may find PyPI down and thus are unable to deploy through the normal fashion.

Finally, there is now a concerted effort to create official mirrors of PyPI that everyone can use (see PEP 381). The mirrors are pretty simple--they're not full blown package indexes--and you can already start using them for experimentation. Also, thanks to Richard Jones, Martin Löwis, Guido, and others at EuroPython, my half-baked idea of using Google App Engine as a PyPI mirror has been resurrected! I don't know if it's fully implemented yet but you can try it out at pypi.appspot.com. App Engine seems like a logical place for a mirror due to its scalability. However, I am skeptical of how well it will perform. App Engine still has a long way to go with regards to stability.

To get involved with the mirroring project you can follow the discussions on the Python Catalog SIG mailing list. The next step will be modifying clients to discover and fallback on available mirrors.

Why you should NOT license your code as GPL

posted in Django, Python, The Future by kumar on Thursday Jul 16th, 2009 at 3:57p.m.

Zed Shaw recently wrote a clear and concise defense for why he used the GPL on Lamson. I've seen a few mentions on twitter that alarmed me because people seemed to think now is the time to release all software as GPL. Here's what you need to ask yourself before you license your code as GPL. First, do you have a business plan that involves selling your software? Most people do not, most business plans have to do with actually using custom software. If that's you then your software and your business become better as more developers work on your software. I.E. when your software is applied to more real world situations, more bugs are fixed and more patches for features are received. GPL does not help you gain users because it shuts out most commercial enterprises ...

The Python Packaging Problem

posted in Python, The Future by kumar on Monday Mar 30th, 2009 at 2:51p.m.

At PyCon 2009 the fact that Python needs to solve the "packaging problem" came up a few times. This is not a new discussion. However, the problem is still not completely solved so here I'll point out the details of the problem, the unsolved parts, the solved parts, and how the solved parts could be solved better ...

Googlebot's Fatal Flaw And How You Can Fix It (or Get Rich Trying)

posted in JavaScript, Python, The Future by kumar on Thursday Feb 12th, 2009 at 2:48p.m.

I came across this article today on Coding Horror about how Google has a monopoly on search engines and how something must be done about it. I'm not one who falls into the "Google Is Evil" camp; I actually think they are a benevolent force in the world :) However, as with any monopoly, the lack of competition stifles progress. And when I think about the state of today's technology, I can't help but wonder why Google has not fixed the most fatal flaw in their Googlebot :

It does not behave like a web browser.

Search engines are made for people and the majority of people browse the Internet with a web browser. The first comment on the article is a cry for help: "What can we do?" I have an answer to that question. And you can take my answer and turn it into a business plan and climb the golden staircase to success. Any smart investor would be begging you to take their money. Google generated $5.37 billion dollars in Q2 of 2008 and their flagship product doesn't even work! In fact, I'm going to give this to you all for free; all I ask is that you visit me one day and say thanks. Are you ready?

Python 3.0 On Mac OS X (alongside 2.6, 2.5, etc)

posted in Python, The Future by kumar on Friday Dec 5th, 2008 at 4:41p.m.

Python 3.0 is out. Woo! Some people have been saying that they'd like to be able to run it alongside 2.6, 2.5, etc to test it out. Well, Python is actually designed for this. All you have to do is build it with make altinstall and that will give you a python3.0 binary without touching your default python binary. But on Mac OS X it's a little different ...

Are you hiring web developers?

posted in Chicago, JavaScript, Python, Ruby, Testing, The Future by kumar on Wednesday Dec 3rd, 2008 at 10:58a.m.

As the US economy spins into a death spiral I unfortunately now know some highly skilled web developers (Python, Ruby, JavaScript, etc) in the Chicago area who are looking for work. And I mean highly skilled people whose ninja like qualities will be sorely missed. Most hiring budgets are probably frozen — at least until next year — but they will soon thaw. Those that thaw sooner will make space for big prizes of talent. I can make introductions over email or via Linked In.

Automated Model Based Testing of Web Applications (GTAC 2008)

posted in GTAC, GTAC 2008, Java, Python, Ruby, Testing, The Future by kumar on Monday Oct 27th, 2008 at 9:36a.m.

Last year at the Google Test Automation Conference (GTAC) 2007, the talk by Atif was one of my favorites. He had been working on a system of model based testing for desktop GUI applications (affectionately called GUITAR) and hinted that he will be applying the work to web applications next. Now, at GTAC 2008, here is a taste of what his dept. has been up to. The talk is by Oluwaseun Akinmade and Prof. Atif M. Memon. Both are at University of Maryland.

The idea of automated model based testing hints at a future where software can be used to figure out how to test itself. That is, when software is modeled in a way that exposes inputs, outputs, event handlers, and end-points, then introspection can be done to find all possible interactions within an application and test them. Yes, it is one step away from artificial intelligence. This is fascinating to me but I think it still needs a lot of work. Atif is asking for as much feedback as possible from industry professionals to find out how this can best be used in the real world.

Here are my notes from the talk ...

Taming The Beast: How To Test an AJAX Application (GTAC 2008)

posted in GTAC, GTAC 2008, JavaScript, Python, Testing, The Future by kumar on Saturday Oct 25th, 2008 at 9:06p.m.

This was one of the talks at GTAC 2008 that I was most looking forward to before the conference. It was excellent, I was not let down. The talk was given by Markus Clermont and John Thomas who work at Google. Since the talk was right after lunch they decided to take a Q & A approach. It sort of went off in tangents at points but overall the format seemed to work.

In my own work I've been struggling at maintaining a now bloated test suite for an AJAX website but their approach made something click in my head. I'm already working on a refactoring plan.

Here is my abbreviated interpretation of the talk ...

The Future of Testing (GTAC 2008)

posted in GTAC, GTAC 2008, Java, JavaScript, Python, Ruby, Testing, The Future by kumar on Friday Oct 24th, 2008 at 3:58a.m.

Google Test Automation Conference (GTAC) is my all-time favorite conference. It's free. It's on a single track — this means you don't miss any talks and everyone experiences the same journey of thought. Also, since you have to apply for admittance with a short essay, everyone who attends is really passionate about testing. It's still sort of "underground" which keeps it small and very social.

Last year, I made some kind of attempt to live blog summaries of the GTAC talks but I never made it past part 1. We'll see how far I get this year, stay tuned.

The videos for 2008 aren't online yet but check youtube often because last year they were up in less than a day.

The Future of Testing was the first talk of the GTAC 2008 conference on Thursday Oct 23rd given by James A. Whittaker, a very entertaining speaker who works for Microsoft. His talk was excellent and I highly recommend keeping a lookout for the video. Here are my notes...

When Online Advertising Actually Works

posted in The Future, Whatever by kumar on Monday Oct 13th, 2008 at 11:04a.m.

Telemarketing is one of the most ineffective forms of advertising. Hello? Yes? I'm cooking dinner, why would I want to buy something? How do you know I'm even remotely interested in your product? Online advertisements like text ads and banner ads are slightly different. I usually buy things online out of convenience so the venue is good for advertising. There is also a lot of information about me online ...

Adrenallin For The Brain

posted in The Future, Whatever by kumar on Saturday Sep 20th, 2008 at 12:56p.m.

Someone passed me a link to this really nice article, Annals of Science: The Eureka Hunt, which talks about how neuroscientists have been studying what goes on in the brain when we get those amazing ideas that seem to come from nowhere. In my own life I can think of several times where I've experienced a sudden "burst" of thought like this. Sometimes it seems like...

It's Time to USE The Web : Mozilla Labs Releases Ubiquity

posted in JavaScript, Python, The Future by kumar on Wednesday Aug 27th, 2008 at 10:26a.m.

Mozilla Labs releases the Ubiquity add-on for Firefox. In a nutshell: With search, users type what they want to find. With Ubiquity, they type what they want to do.

Big congrats to Aza and everyone else who worked on this! As innovative as Humanized was, building something similar into Firefox is genius. Personally I am excited by the prospect of developing work flows on the Ubiquity platform. There are so many common work flows that need simplification.

Developers: If you want to cut to the chase, Ajaxian shows how easy it is to create custom commands. And of course, the details are in the Authoring Tutorial.

Making Erlang indentation-sensitive

posted in Erlang, Python, The Future by kumar on Monday Apr 28th, 2008 at 5:56p.m.

Ulf Wiger has been experimenting with making Erlang indentation-senstive. He followed up with a Part 2 after comments.

For most people who start working in the Erlang language, the most annoying new concept is all the various line-terminators you have to know. Damien Katz sums this up nicely. All these explicit terminators might make other people's code easier to read but writing code should be easy too (they are equally important). Mue says just get over it but as a Python user I agree that Erlang would be way better off if it was indentation-based so this was an interesting experiment.

As for Python, I'm still shocked to hear people say "Python would be nice ... if it wasn't for that whitespace thing." Who is perpetuating this? I have no idea. The next time you hear someone say this, pass along Python: Myths about Indentation!

PyPi (Cheeseshop) on Google App Engine

posted in Google App Engine, Projects, Python, The Future by kumar on Tuesday Apr 15th, 2008 at 9:55a.m.

Like many of you, I've had my jaw on the floor since the release of Google App Engine. Although there are skeptics out there, a careful read of their terms will show you that it's for real — Google has released GOOGLE to the world and it's not for scary marketing purposes. In fact, I've been growing tired of paranoid Google haters; I'm hoping this will shut them up for a while.

Why is App Engine such a breakthrough? The concept of a hosted web application is nothing new but it has never been done this well. Mundane server maintenance? Gone. Infinite scalability? Check. 100% uptime? Let's face it, if Google went offline you'd probably be down in a nuclear bunker playing Parcheesi.

So ... how should we leverage this tool for the greater good of the community? I can't count the ways without getting dizzy. How about let's start with a mirror of PyPi, the Python Package Index?

PyPi on the App Engine

I barely spent two days on it, but here it is: http://pypi.appspot.com/. Test it out, play with it, try to break it.

As Python grows, especially due to App Engine, PyPi needs to scale too. Zope has put together a PyPi mirror but that's the only other one I know of (actually, I can't even find the link to it right now). Coincidentally, PyPi even went offline for a few min while I was writing this blog post.

Issues...

You Can Help

I'm not dedicated to this project, I just thought it sounded like a good idea and would be a fun way to experiment with the App Engine. If anyone is interested in working on it just let me know --kumar.mcmillan@gmail.com. If there is enough interest I'll put it on Google Code. Possibly the most exciting feature of App Engine is the Datastore API (aka BigTable) and Ben Bangert agrees. It's a little hard for me to wrap my head around it but so far the Expando class—besides being the coolest name for a class—seems to work great for storing package data. If EGG-INO grows a new parameter, it just gets tacked on to the row dynamically.

This has also been a great way to dig up bugs, some of which have already been fixed.

Software is written by hand

posted in JavaScript, Python, Ruby, The Future, Whatever by kumar on Friday Dec 21st, 2007 at 3:36p.m.

...that's right, it's not molded or prefabbed, it's not made on a production line or in a lab. Are we insane??! Here is a hilarious probe into the darker side of this art we call programming.

How To Get Started Writing Open Social Applications

posted in JavaScript, Presentations, Python, The Future by kumar on Sunday Nov 18th, 2007 at 11:58a.m.

GTAC Highlights Part 1 - Selenium is Alive and Well, Model Based Testing Is Smart, And...

posted in GTAC, Java, Python, Ruby, Testing, The Future by kumar on Saturday Aug 25th, 2007 at 5:34p.m.

Humans are here to stay!

posted in The Future by kumar on Wednesday Apr 4th, 2007 at 10:15p.m.

Why People Don't Use Hand Dryers

posted in Python, The Future by kumar on Wednesday Feb 14th, 2007 at 10:32a.m.

Recent Projects

  • Fudge

    Replace real objects with fakes when testing.

  • Fixture

    Loading and referencing test data.

  • NoseJS

    Nose plugin that runs JavaScript tests for a Python project.

  • Wikir

    converts reST to various Wiki formats.