Yesterday, Atul Varma announced he had resurrected python-spidermonkey, John J. Lee's project to run JavaScript in Python. Woo! Spider Monkey is the JavaScript runtime that Mozilla (i.e. Firefox) uses internally. However, it's just an engine and doesn't deal with the DOM or anything about the typical web browser environment JavaScript files usually run in. Roughly, that environment defines the following JavaScript objects:
- document
- window
- navigator
- layer
- and all the form objects
I've always been interested in python-spidermonkey for the prospect of fully testing JavaScript-dependent web applications in Python alone. This is no small feat and there are many caveats. The first caveat is that if the Application Under Test is running in something too far off from its native runtime then the value of your tests rapidly decreases. In other words, the best way to test a JavaScript-heavy web app is to use Selenium or Windmill to automate and introspect the app in its native runtime: the web browser. I believe Java has Rhino but since it's implemented in Java my guess is its behavior is pretty far off from that of Firefox. We all know and love the quirks of web browsers; this is what makes web development so fun :(
A lightweight version of Firefox — its JavaScript engine, its DOM, etc — could possibly make web testing a lot more efficient. By lightweight I mean no GUI, no visual rendering (although removing this may greatly hinder JavaScript, not sure). Why? At the company I'm at we build and test web applications. Our largest suite of Selenium tests that automate a real web browser takes about 2 hours to run and that's expensive. We are looking into a Selenium Grid solution but if there's a way to simply cut out all the browser overhead that we don't need, this would be a huge score.
And, oh yeah, I haven't even talked about how tests for web apps need to run in Internet Explorer if they are to be worth anything. However, we make some apps for internal company use and thus can mandate the use of Firefox. To solve the cross-browser problem one approach could be to implement all this logic as a driver that can be hooked into the Selenium interface we use, which is all in Python code thanks to Selenium Remote Control. This the exact approach that Webdriver for Java takes &mdash you can simply tell it to use a faster in-process web browser and it will run JavaScript but only via Rhino, which goes back to the non-native runtime problem. In fact, WebDriver will soon be an actual driver for Selenium.
Hmm ... thinking out loud ... maybe the solution is to just run XULRunner as a headless firefox, the way SMILE web scraper does. Maybe I'll try that.
Besides web testing, one cool thing about python-spidermonkey is it may allow for unit testing of JavaScript libs in Python. This means you could create an import hook that loaded *.js files into Python and do all your testing in Python, reaping the benefits of tools like nose. Or, maybe more realistically, automate the testing of custom Firefox extensions implemented in JavaScript.
Have you ever ran unit tests in JavaScript? It's clunky because you need to run them in a browser. That's how JSUnit works anyway, which is otherwise a passable testing environment. To automate such tests you have to use something like Selenium to run your test suite in a real web browser.
Re: An In-Process, Headless Web Browser for Python?
posted by Michael Foord on Thursday Jun 12th, 2008 at 3:58p.m.
This will be *great* for writing email harvesting web spiders that understand Javascript! :-)
Re: An In-Process, Headless Web Browser for Python?
posted by Kumar McMillan on Thursday Jun 12th, 2008 at 4:52p.m.
Heh, maybe that's what SMILE was built for
Re: An In-Process, Headless Web Browser for Python?
posted by Curtis Thompson on Friday Jun 13th, 2008 at 10:45a.m.
In Rhino 1.7R2 Norris Boyd implemented doctest for javascript. That doesn't help with the dom stuff, but is pretty cool nevertheless.
For the dom stuff, we have a 'Fake Mozilla' that we've implemented in Java. It still needs a lot of work, so it would be nice to open that up as an open source project.
We're writing a FireFox add-on, so we use MozRepl (similar to JSSh) a lot and we've been thinking about using that with a headless x11 session to drive tests. These work ok.. but we're still looking for something better. :-)
Re: An In-Process, Headless Web Browser for Python?
posted by Roman Ziak on Saturday Oct 11th, 2008 at 5:30p.m.
This reminds me an instance when we had internal application that was endlessly scrolling the ListBox (Windows).
The redrawing was taking significant amount of time, so we disabled it during certain operations:
SendMessage(hwnd, WM_SETREDRAW, FALSE, 0);
then after we were done, we enabled it back:
SendMessage(hwnd, WM_SETREDRAW, TRUE, 0);
InvalidateRect(hwnd, NULL, TRUE);
and the speed-up was substantial.
Another option would be to hide the window, low-level Windows GDI will speed-up significantly when calling drawing primitives in the window that is not showing.
Hope this helps.
Re: An In-Process, Headless Web Browser for Python?
posted by eBook Search on Wednesday Jul 1st, 2009 at 7:13a.m.
My refusal to use Windows is what makes me learn to do everything using only Linux, even though it usually requires long times in front of the computer researching and tweaking the hints google.com/linux gives me. But, at least I can say that at the end the results are satisfying, I end up learning more about Linux (recommend start from the ebook http://www.ebook-search-queen.com ) and how powerful a command line application can be compared to a most GUI application. I must admit that I do prefer GUI in the majority of the cases, is just that from time to time I am amazed by the power of the CLI.
Re: An In-Process, Headless Web Browser for Python?
posted by personal injury fort lauderdale on Tuesday Aug 16th, 2011 at 2:53a.m.
I don't read about python before.I didn't know that Python is a programming language,Thanks for knowing me.personal injury fort lauderdale
Re: An In-Process, Headless Web Browser for Python?
posted by personal injury fort lauderdale on Tuesday Aug 16th, 2011 at 3:56a.m.
Impressive blog ,I never read this types of blog before. thanks to providing this type of interesting content. I’ll be waiting for your next blog.
Re: An In-Process, Headless Web Browser for Python?
posted by Transport Boat on Monday Sep 12th, 2011 at 7:58a.m.
Hello My Dear Friend , The post is written in very a good manner and it entails many useful information for me. I appreciated what you have done here. I am always searching for informative information like this. Thanks for sharing with us.
Re: An In-Process, Headless Web Browser for Python?
posted by Auto Shipping Maryland on Monday Sep 19th, 2011 at 12:45a.m.
Hello My Dear Friend , The post is written in very a good manner and it entails many useful information for me. I appreciated what you have done here. I am always searching for informative information like this. Thanks for sharing with us.
Re: An In-Process, Headless Web Browser for Python?
posted by lamborghini rent miami on Friday Oct 21st, 2011 at 12:58a.m.
This is a great inspiring article. I am pretty much pleased with your good work. You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post.
Re: An In-Process, Headless Web Browser for Python?
posted by computer repair maintenance on Friday Oct 21st, 2011 at 4:23a.m.
Useful information shared..Iam very happy to read this article. Thanks for giving us nice info. Fantastic walk-through. I appreciate this post.
Re: An In-Process, Headless Web Browser for Python?
posted by computer repair maintenance on Friday Oct 21st, 2011 at 4:24a.m.
Useful information shared..Iam very happy to read this article. Thanks for giving us nice info. Fantastic walk-through. I appreciate this post.
Re: An In-Process, Headless Web Browser for Python?
posted by hämorrhoiden behandlung on Friday Nov 18th, 2011 at 4:43p.m.
I was looking for something like this…I found it I am bookmarking this and sharing with my friends on facebook, twitter and my space.
Re: An In-Process, Headless Web Browser for Python?
posted by SEO Services on Tuesday May 8th, 2012 at 9:04a.m.
Really it is a great post in all. I got authority for this article in this site. Because i read this post attentively and acquire some useful information.
SEO Services