python-twitter vs Google App Engine

In preparation for tackling Wazup, I started learning a little bit about Python (I know many people have said this already, but Dive Into Python is a really good book!) and I decided to try out a much smaller (and hopefully easier) app to begin with.
So I began coding a little toy that does some stuff with my twitter friends list.

I’m using python-twitter and the app is coming along nicely, except…

Well, like I said earlier, I’m something of an old-hand at programming and my first instinct when learning a new language or tackling a new kind of problem is to first get it working on the command-line and then get the code to work on the web (or whatever happens to be my intended platform). Yes, and my IDE is vim OK?

So what’s the big deal, you may ask?

When I ported my app, which was running perfectly on the command-line, into the App Engine development platform it started bombing, complaining about not being able to get the current user.

Say what?? Well, one thing I’m more than used to is to debug stuff, even in languages I don’t master. And this was very easy to do actually, so I got to the point really fast: python-twitter assumes you’re running on a real machine and implements a caching mechanism that assumes you have such things as a tmp dir where you can store your temporary files.
The code was trying to determine who I was in order to find out where it should store it’s temporary cache files.

Of course in a sandboxed environment such as the App Engine, there is no such thing as disk-based storage (or even system users for that matter), so it was failing miserably.

Turns out that someone else had already been there and the way to work around this is to modify python-twitter and make it use App Engine’s own datastore for the caching. That is if I do think it needs caching at all, I’m still not too sure about that.

Oh well, my first project to “ease” myself \into the language and already I’m making changes to other people’s code…

It’s been fun, though, I must admit to that!

Going to the doctor is good for you

In more ways than one…

Today I went to the doctor for my biyearly allergies checkup and, as usual, I got stuck in the waiting room for hours.

The good part about it was that I took my laptop and cell phone with me and I ended up finishing the serializer part of the Flickr-Tools project.
It can now get the full photo collection of a user, in chunks, and give it out as a serializer to the user, without him being aware that much fetching is (potentially) going on in the background.

Now all that I’m lacking is the support for the new authentication method (essentially I need to read up on the whole frob passing stuff and I need a way for users to do the first authentication) and I’ll release it to the world.

What’s that I hear? “But wasn’t that supposed to be done weeks ago?” well yes but you know, I love the swoshing sound of deadlines going past so much that I’d do just about anything to get it… :-)

Darcs projects site

Trying to not get too stale on my developments (OK, I’m way beyond that, I know…) and because people are actively asking me to add features to the Flickr-Tools distribution and, better yet, some are actually offering to send in patches (hurra!) I’ve decided it was high time I put up some sort of project archive on-line.

So after a little tinkering, darcs.nunonunes.org is now on-line and open for business, default css and all!

In the future I might setup a trac site also but for now darcsweb is what I have available.

So now that you know where to get it go crazy with the patches!

(Actually this is a few days old, but I never got around to posting it…)

An easier Flickr API in Perl

Sooo… Last night Tuxa went to a farewell dinner she and her friends threw for a colleague and I was left home alone without the usual many urgent things to do, so I took a stab at the Flickr modules I’m starting to write.

And now I’m trying to decide which is the better approach to take.

One possible approach would be to create a Flickr object which has a method for each of the Flickr API calls, such as $flickr->photosgetInfo_ or $flickr->peoplefindByEmail_ and so on.
The object should then return a hash structure that closely resembles the documented result for that call, so for example the $flickr->photosgetInfo_ method (described here) should return something along the lines of the following:

{
  id => "2733",
  secret => "123456",
  server => "12",
  isfavorite => "0",
  license => "3",
  rotation => "90",
  owner => {
             nsid => "12037949754@N01",
             username => "Bees",
             realname => "Cal Henderson",
             location => "Bedford, UK",
           },
  title => "orford_castle_taster",
  description => "hello!",
  visibility => {
                  ispublic => "1",
                  isfriend => "0",
                  isfamily => "0",
                },
  dates => {
             posted => "1100897479",
             taken => "2004-11-19 12:51:19",
             takengranularity="0",
           },
  permissions => {
                   permcomment="3",
                   permaddmeta="2",
                 },
  editability => {
                   cancomment="1",
                   canaddmeta="1",
                 },
  comments => 1,
  notes => [
             {
               id => "313",
               author => "12037949754@N01",
               authorname => "Bees",
               x => "10",
               y="10",
               w="50",
               h="50",
               content => "foo",
             },
           ],
  tags => [
            {
              id => "1234",
              author => "12037949754@N01"
              raw => "woo yay",
              content => "wooyay",
            },
            {
              id => "1235",
              author => "12037949754@N01",
              raw="hoopla",
              content => "hoopla",
            },
          ],
}

This kind of approach appeals to me because it gives me access to just the plain data, in a easy way, without all the XML-parser-generated-cruft, in a much more “perlish” way.

But while I consider this an improvement over the current Flickr::API module (which is great, by the way, and will be used as the foundation for all of this), it may still be too low-level for many people to use.

So there is another possible approach (OK, there are many other possible approaches but I’m focusing one on in particular) ;-) which is to create a set of higher-level objects which represent an entity like a “person” or a “photo” or a “photoset”.

Then the user would simply ask the object to instantiate, say, the Flickr user whose email is something@somesuch (using flickr.people.findByEmail on the background) or the photo whose id is someid (using flickr.photos.getInfo). Then, if the user wanted some more information on the photo, for example, she could ask for the lens aperture used ($photo->apperture()) and the object would fetch the EXIF information for the photo in question (using flickr.photos.getExif), load it into the $photo object and return the information.
A major point to make here is that the object would fetch information in a “lazy” manner, so as to limit the number of interactions it has to have with the Flickr server.

My immediate needs are more than adequately met by the first approach to the modules and, therefore, I guess I’ll surely implement them right now, but the second approach could be very interesting indeed…
But then, now that I think about it in a more structured way, the second way depends on the first one to process all the results from the raw API and convert them into useful data, so I may well end up implementing it too in the future…

What I haven’t figured out just yet is which name-space to use.
Should I go for Flickr::API::Methods (as in Flickr::API::Methods::peoplefindByEmail()_ or Flickr::API::Methods::photosgetInfo())?
Or maybe go for _Flickr::Tools…
Or I could go with Flickr::Photos (Flickr::Photos::getInfo() for instance), Flickr::People (Flickr::People::findByUsername()). Actually I think I like this last version best…

As for the higher-level modules I’m not entirely convinced they should even be in the Flickr namespace at all… But then I don’t have to decide on it just yet so let’s see what my mind is aimed at when I get to it.

If anyone reading this has any kind of comment, remark, idea, suggestion or tough about the subject I really would appreciate feedback. (Hey, I guess this is the first time I explicitly ask for feed-back here… Let’s see how it goes. I should probably start using the use Perl; journal —which I’ve never used at all— for this though… maybe I’ll cross-post there.)

Technorati Tags: Coding, Perl

MP3Slicer — cut up your podcasts

[Update (June 23rd, 2009)]: This script has been removed during a site clean-up.

So today I spent almost two hours waiting in the lounge at the doctor’s office. Quite usual, as a matter of fact but this time I decided to get the lemons life gave me and make some lemonade (as it where…)

And as I happened to have some podcasts just waiting to be recorded on a CDRW to take to my car I decided to try and finish up a little script I’ve been trying to put together for a while.

As it turns out I nailed it today before I got my allergies checked up and while I can’t say I’m terribly proud of the job I’ve made (so far) it does do the job, so you can find my MP3Slicer script here (at least for the time being).

As I state in the scripts page I just had to cut up the podcasts into manageable chunks to make it practical to listen to them while on the car or at the gym and this is my first stab at it.

I can’t remember how long I’ve been trying to get this done so the long wait turned out to be a good productivity enhancer. Who’d have thunk it, heh? ;-)

Syndigator in numbers

Since my conversion I’ve effectively stopped using Syndigator but since this project is still close to my heart (and because I’m a friend of the developers - this helps a lot, of course) I keep a close watch on what’s going on with it and even do some of the boring tasks related to new releases (like dealing with the freshmeat page) out of the kindness of my heart… :)

Syndigator is about to have a new release which will mark the port to Gtk2 and I went around to it’s project page on SourceForge and, while I was at it, took a look at the statistics page.

Now this project is totally and completely based on it’s authors’ free time and is, in fact, a hobby of theirs. So are a lot of other open-source projects. But in this case there has been close to no effort spent in publicising it to the world at large.
There have been some initial posts on some weblogs, there is the mandatory Freshmeat page and some word-of-mouth was used, but other than that (and after the initial surge) things have mostly been very quiet on the “trumpet blowing” front.

So why all this now? Because what I saw on the numbers is quite intriguing (for me at least) and bodes very well indeed for the whole venture.
The project is close to one year old and apart from the total redesign of it’s web page (kindly maintained by a most welcome volunteer) there haven’t been any major changes to it and still we are steadily above the 6.000 pageviews/month and have been above the 10.000 mark since December 2003.
Now this may not seem to be very relevant but when I look at the number of downloads per month I see that we are above 400 a month since January 2004 and steadily rising. On the last days we have been well over 10 downloads a day. Now remember, nothing new has happened in terms of releases for quite some time now and absolutely no effort has been made from any of us in order to make Syndigator more widely known.

This leads me to the conclusion that the software is indeed:
a) good software
b) useful
c) better than the alternatives

Don’t get me wrong, I know these figures are puny on an all-time great-opensource-software scale, but for such a modest project to fair this well is something very enjoyable to watch indeed.

Kudos to Bruno and Paulo (who has to get a page on the web for pity’s sake!) ;) for a good job!

Syndigator sailing along

Another release with some new features.
Bruno has come from his vacation and has been busy getting some new features in Syndigator.
Also he is quite advanced in the Gtk2 port of it so expect another release soon enough with Gtk2 support but as yet without Gtk-Html support as it is not yet ported to Perl.
I have been on vacation also and didn’t do a thing about the cocoa port so it is something I will do, only I don’t know when… :)

Syndigator - another release

Another minor release of syndigator. Read about it here.
Also, I’m asking for suggestions and opinions on there to add to syndigator.

It is really difficult for me to think of things which are both useful and easy to use.
Mostly I just want stuff that is either really complicated to develop (like synchronizing multiple “frontends” with a common “storage” and “state maintainer”) or that requires resources not commonly available to the average user (like having access to your own always-on server with a database for example).
So while I still want the “geek” features to be implemented, they will take some time to think about and to develop and in the mean time I’d like to add some more features that a regular user would like to have.

Particularly difficult for me is the user interface and user-friendly part of it. Well, not to me exactly, since I’m not the Gtk programmer, but to the development group on the whole.
We are all geeks, and that makes our judgment of a user-interface skewed to say the least. Also, the fact that we have watched this product grow makes us fully aware of it’s behavior and all the reasons for it to be like it is so for us it is not easy to think “what would the average user think about this” or “where would an average user expect to find a button that would do this or that”…

Oh well, suggestions are very welcome in the syndigator blog.

[The links mentioned on the previous paragraphs were removed because the site they referenced no longer exists.]