Thoughts on The Future
Python Package Index (PyPI) Mirrors
Most deployment systems depend on PyPI, the Python Package Index, for fetching and installing dependencies. Although performance and reliability has greatly improved, there are still days when you may find PyPI down and thus are unable to deploy through the normal fashion.
Finally, there is now a concerted effort to create official mirrors of PyPI that everyone can use (see PEP 381). The mirrors are pretty simple--they're not full blown package indexes--and you can already start using them for experimentation. Also, thanks to Richard Jones, Martin Löwis, Guido, and others at EuroPython, my half-baked idea of using Google App Engine as a PyPI mirror has been resurrected! I don't know if it's fully implemented yet but you can try it out at pypi.appspot.com. App Engine seems like a logical place for a mirror due to its scalability. However, I am skeptical of how well it will perform. App Engine still has a long way to go with regards to stability.
To get involved with the mirroring project you can follow the discussions on the Python Catalog SIG mailing list. The next step will be modifying clients to discover and fallback on available mirrors.
- read article
- 0 comments
- read comments
- post comment
Why you should NOT license your code as GPL
Zed Shaw recently wrote a clear and concise defense for why he used the GPL on Lamson. I've seen a few mentions on twitter that alarmed me because people seemed to think now is the time to release all software as GPL. Here's what you need to ask yourself before you license your code as GPL. First, do you have a business plan that involves selling your software? Most people do not, most business plans have to do with actually using custom software. If that's you then your software and your business become better as more developers work on your software. I.E. when your software is applied to more real world situations, more bugs are fixed and more patches for features are received. GPL does not help you gain users because it shuts out most commercial enterprises ...
- read article
- 12 comments
- read comments
- post comment
The Python Packaging Problem
At PyCon 2009 the fact that Python needs to solve the "packaging problem" came up a few times. This is not a new discussion. However, the problem is still not completely solved so here I'll point out the details of the problem, the unsolved parts, the solved parts, and how the solved parts could be solved better ...
- read article
- 12 comments
- read comments
- post comment
Googlebot's Fatal Flaw And How You Can Fix It (or Get Rich Trying)
I came across this article today on Coding Horror about how Google has a monopoly on search engines and how something must be done about it. I'm not one who falls into the "Google Is Evil" camp; I actually think they are a benevolent force in the world :) However, as with any monopoly, the lack of competition stifles progress. And when I think about the state of today's technology, I can't help but wonder why Google has not fixed the most fatal flaw in their Googlebot :
It does not behave like a web browser.
Search engines are made for people and the majority of people browse the Internet with a web browser. The first comment on the article is a cry for help: "What can we do?" I have an answer to that question. And you can take my answer and turn it into a business plan and climb the golden staircase to success. Any smart investor would be begging you to take their money. Google generated $5.37 billion dollars in Q2 of 2008 and their flagship product doesn't even work! In fact, I'm going to give this to you all for free; all I ask is that you visit me one day and say thanks. Are you ready?
- read article
- 8 comments
- read comments
- post comment
Python 3.0 On Mac OS X (alongside 2.6, 2.5, etc)
Python 3.0 is out. Woo! Some people have been saying that they'd like to be able to run it alongside 2.6, 2.5, etc to test it out. Well, Python is actually designed for this. All you have to do is build it with make altinstall and that will give you a python3.0 binary without touching your default python binary. But on Mac OS X it's a little different ...
- read article
- 42 comments
- read comments
- post comment
Are you hiring web developers?
As the US economy spins into a death spiral I unfortunately now know some highly skilled web developers (Python, Ruby, JavaScript, etc) in the Chicago area who are looking for work. And I mean highly skilled people whose ninja like qualities will be sorely missed. Most hiring budgets are probably frozen — at least until next year — but they will soon thaw. Those that thaw sooner will make space for big prizes of talent. I can make introductions over email or via Linked In.
- read article
- 4 comments
- read comments
- post comment
Automated Model Based Testing of Web Applications (GTAC 2008)
Last year at the Google Test Automation Conference (GTAC) 2007, the talk by Atif was one of my favorites. He had been working on a system of model based testing for desktop GUI applications (affectionately called GUITAR) and hinted that he will be applying the work to web applications next. Now, at GTAC 2008, here is a taste of what his dept. has been up to. The talk is by Oluwaseun Akinmade and Prof. Atif M. Memon. Both are at University of Maryland.
The idea of automated model based testing hints at a future where software can be used to figure out how to test itself. That is, when software is modeled in a way that exposes inputs, outputs, event handlers, and end-points, then introspection can be done to find all possible interactions within an application and test them. Yes, it is one step away from artificial intelligence. This is fascinating to me but I think it still needs a lot of work. Atif is asking for as much feedback as possible from industry professionals to find out how this can best be used in the real world.
Here are my notes from the talk ...
- read article
- 4 comments
- read comments
- post comment
Taming The Beast: How To Test an AJAX Application (GTAC 2008)
This was one of the talks at GTAC 2008 that I was most looking forward to before the conference. It was excellent, I was not let down. The talk was given by Markus Clermont and John Thomas who work at Google. Since the talk was right after lunch they decided to take a Q & A approach. It sort of went off in tangents at points but overall the format seemed to work.
In my own work I've been struggling at maintaining a now bloated test suite for an AJAX website but their approach made something click in my head. I'm already working on a refactoring plan.
Here is my abbreviated interpretation of the talk ...
- read article
- 1 comment
- read comments
- post comment
The Future of Testing (GTAC 2008)
Google Test Automation Conference (GTAC) is my all-time favorite conference. It's free. It's on a single track — this means you don't miss any talks and everyone experiences the same journey of thought. Also, since you have to apply for admittance with a short essay, everyone who attends is really passionate about testing. It's still sort of "underground" which keeps it small and very social.
Last year, I made some kind of attempt to live blog summaries of the GTAC talks but I never made it past part 1. We'll see how far I get this year, stay tuned.
The videos for 2008 aren't online yet but check youtube often because last year they were up in less than a day.
The Future of Testing was the first talk of the GTAC 2008 conference on Thursday Oct 23rd given by James A. Whittaker, a very entertaining speaker who works for Microsoft. His talk was excellent and I highly recommend keeping a lookout for the video. Here are my notes...
- read article
- 3 comments
- read comments
- post comment
When Online Advertising Actually Works
Telemarketing is one of the most ineffective forms of advertising. Hello? Yes? I'm cooking dinner, why would I want to buy something? How do you know I'm even remotely interested in your product? Online advertisements like text ads and banner ads are slightly different. I usually buy things online out of convenience so the venue is good for advertising. There is also a lot of information about me online ...
- read article
- 2 comments
- read comments
- post comment
Adrenallin For The Brain
Someone passed me a link to this really nice article, Annals of Science: The Eureka Hunt, which talks about how neuroscientists have been studying what goes on in the brain when we get those amazing ideas that seem to come from nowhere. In my own life I can think of several times where I've experienced a sudden "burst" of thought like this. Sometimes it seems like...
- read article
- 1 comment
- read comments
- post comment
It's Time to USE The Web : Mozilla Labs Releases Ubiquity
Mozilla Labs releases the Ubiquity add-on for Firefox. In a nutshell: With search, users type what they want to find. With Ubiquity, they type what they want to do.
Big congrats to Aza and everyone else who worked on this! As innovative as Humanized was, building something similar into Firefox is genius. Personally I am excited by the prospect of developing work flows on the Ubiquity platform. There are so many common work flows that need simplification.
Developers: If you want to cut to the chase, Ajaxian shows how easy it is to create custom commands. And of course, the details are in the Authoring Tutorial.
- read article
- 0 comments
- read comments
- post comment
Making Erlang indentation-sensitive
Ulf Wiger has been experimenting with making Erlang indentation-senstive. He followed up with a Part 2 after comments.
For most people who start working in the Erlang language, the most annoying new concept is all the various line-terminators you have to know. Damien Katz sums this up nicely. All these explicit terminators might make other people's code easier to read but writing code should be easy too (they are equally important). Mue says just get over it but as a Python user I agree that Erlang would be way better off if it was indentation-based so this was an interesting experiment.
As for Python, I'm still shocked to hear people say "Python would be nice ... if it wasn't for that whitespace thing." Who is perpetuating this? I have no idea. The next time you hear someone say this, pass along Python: Myths about Indentation!
- read article
- 5 comments
- read comments
- post comment
PyPi (Cheeseshop) on Google App Engine
Like many of you, I've had my jaw on the floor since the release of Google App Engine. Although there are skeptics out there, a careful read of their terms will show you that it's for real — Google has released GOOGLE to the world and it's not for scary marketing purposes. In fact, I've been growing tired of paranoid Google haters; I'm hoping this will shut them up for a while.
Why is App Engine such a breakthrough? The concept of a hosted web application is nothing new but it has never been done this well. Mundane server maintenance? Gone. Infinite scalability? Check. 100% uptime? Let's face it, if Google went offline you'd probably be down in a nuclear bunker playing Parcheesi.
So ... how should we leverage this tool for the greater good of the community? I can't count the ways without getting dizzy. How about let's start with a mirror of PyPi, the Python Package Index?
PyPi on the App Engine
I barely spent two days on it, but here it is: http://pypi.appspot.com/. Test it out, play with it, try to break it.
As Python grows, especially due to App Engine, PyPi needs to scale too. Zope has put together a PyPi mirror but that's the only other one I know of (actually, I can't even find the link to it right now). Coincidentally, PyPi even went offline for a few min while I was writing this blog post.
Issues...
- urlfetch can't get big packages
- fetching SQLAlchemy triggered the ResponseTooLarge error but its package was barely over 1MB. We might be able to create a download proxy to work around this.
- May not be fully compatible with easy_install yet
- Can't run scripts on App Engine
- Not too big a deal, you have to think of web pages as scripts and use a separate crontab to make timed requests. There are security issues with this, but for mirroring pypi it's not a problem
- Note that I haven't actually mirrored all of PyPi, I'm still testing it out.
- It's not very pretty yet (see below)
- The prototype works, now it needs some tests :)
You Can Help
I'm not dedicated to this project, I just thought it sounded like a good idea and would be a fun way to experiment with the App Engine. If anyone is interested in working on it just let me know --kumar.mcmillan@gmail.com. If there is enough interest I'll put it on Google Code. Possibly the most exciting feature of App Engine is the Datastore API (aka BigTable) and Ben Bangert agrees. It's a little hard for me to wrap my head around it but so far the Expando class—besides being the coolest name for a class—seems to work great for storing package data. If EGG-INO grows a new parameter, it just gets tacked on to the row dynamically.
This has also been a great way to dig up bugs, some of which have already been fixed.
- read article
- 6 comments
- read comments
- post comment
Software is written by hand
...that's right, it's not molded or prefabbed, it's not made on a production line or in a lab. Are we insane??! Here is a hilarious probe into the darker side of this art we call programming.
- read article
- 0 comments
- read comments
- post comment
How To Get Started Writing Open Social Applications
- read article
- 1 comment
- read comments
- post comment
GTAC Highlights Part 1 - Selenium is Alive and Well, Model Based Testing Is Smart, And...
- read article
- 11 comments
- read comments
- post comment
Humans are here to stay!
- read article
- 0 comments
- read comments
- post comment
Why People Don't Use Hand Dryers
- read article
- 1 comment
- read comments
- post comment
