It’s definite now: I’m going to the London Perl Workshop.

And not only that, but I’m going a couple of days earlier (I’ll have time to walk around London again, as I’ve been craving to do for a while), I’m going with friends (party!) :-) and I’m staying at a friend’s (no hotel fee).

Yay!

This year’s London Perl Workshop will be on December first.

Tempting, very tempting… I wonder if I could find somewhere to crash for a couple of nights for free… Hum…

Today’s lesson is manyfold:

1. Devel::Size is your friend;
2. Maintaining state across runs of a program is good unless said state is able to keep getting bigger because of circumstances beyond your control;
3. In which case it is a very good idea to keep checking it for size and pruning it whenever you have to…
4. Especially if you launch many many copies of said program in parallel.

Got all that? Good! :-)

As I mentioned yesterday, a few packages caught my attention during the last few days at YAPC and yesterday during the afternoon and evening I took advantage of it being a lighter day (conference-wise) and decided to check out two of them.

First off a took a look at Plagger. Now instead of using the handy pre-packaged versions out there, I went the “fanatic CPAN hardcore user” way (as the author puts it) and let me tell you, you have to have at least some degree of fanaticism to go through with it alright! Oh and it is true what was said during the conference: installing Plagger —the CPAN way— sure does keep your computer warm! :-)

But hey, I got through with it and then it was all a matter of trying out a few configuration recipes to get the hang of things and after that mashing some options together to see how it all worked out.

And let me tell you, it works out beautifully! Plagger is so powerful it is hard to describe and the plugin system wins out yet again. You can setup an aggregator to feed you news and events to your email in a couple of minutes, full of bells and whistles. Oh and doing a “plannet” type of site couldn’t be easier! Put that together with a myriad notification systems (IRC, Growl, …) and a powerful filtering system and, well, it really rocks!

The only problem I see with it is that you do have to occasionally look at the source code of some modules to really understand what’s going on there and what you can actually do with them. Documentation is something that this project sorely needs and I fear that until there is such documentation it will continue to be mostly the province of Perl hackers!

If you wish to take a look at the presentation on the subject given at YAPC, you can find it here and be sure to check out the neat examples which work (almost!) out of the box.
Also check out the architecture of the thing, it really helps you figure out how to use things to create the specific aggregator you wish.

The other piece of Perl I tried out yesterday was KinoSearch.

I’ve had the need for search on a site come up a little while ago but hadn’t yet had the opportunity of going through the available modules on CPAN and now it seems like I won’t need to. Yay YAPC! :-)

I was fortunate enough to attend two talks on the subject and from the first one I got the feeling that there were two packages which could possibly interest me —Search::Xapian and KinoSearch. It seems like Search::Xapian is the more powerful of the two but it lacks in documentation and is really hard to use.

Then there was the second talk which focused specifically on KinoSearch and I was amazed at what I saw there, so I decided to go and try it out.

Installing it was a breeze (straight out of CPAN, of course) and using it was ridiculously easy!
I mean, I just copied and pasted the examples straight out of perldoc, tweaked what I needed to in order for my case to work (fortunately I have a fairly up-to-date copy of the site on my laptop), ran it and voilá! A fully working (fast!) index of the whole site!

Now of course they were right in the presentation when they said it can be a bit of a hog when it comes to disk space —it took 2.6M to index a full 2.7M of text!!— but then I used it straight out of the examples so maybe I can get it to be a bit more civil disk space-wise with a bit more tweaking…

And now to get some breakfast, pack my stuff and go out to see a bit more of the city until it is time to get on the train to Stratford for the rest of the vacation. Oh I see it’s raining quite a bit out there… How very typical! :-)

This is just a quick post with a list of cool stuff I found out about (or was reminded of) during YAPC::EU::2006 and will want to review later, in no particular order.

• Parrot is coming along quite nicely to the point where there is now some sort of light at the end of the tunnel;
• CPANPlus
• Searching with Perl is hot right now - Search::Xapian and KinoSearch
• Test::Class rules (re-read Perl Testing book, chapter 8)
• Test::WWW:::Selenium looks very nice
• Plagger is looking very, very good!

Lots more to report when I digest my notes but this is the list that comes to mind right now.

Technorati Tags: Perl, YAPC

Sheesh, it just never stops these days! :-)

Well, but in this case I’m more of a helper/facilitator than an actual regular poster.

So without further ado, ladies and gentleman: Today was the official launch of the Lisbon Perl Mongers’ blog at lisbon.pm.org/blog.
This blog will be dedicated to news and activities of the group and therefore, the content will be mainly in Portuguese with possibly a few posts in English (when it is justifiable).

There’s not much there yet, but our fearsome leader will surely fill it to the brim with usefull news and bits of information, so stay tuned!

Technorati Tags: Perl, Lisbon.pm

After having pushed myself to work a bit harder on the Flickr-Tools project and trying to get a new version out before the end of January (which is shot, by the way), I finally have a preliminary version of the Flickr::Person::photos() method working! Urrah!

So you can now use this method and get back the first 100 photos of a particular person. After that first batch of 100 photos it just stops giving you new stuff and closes up shop but it is rather nice to have it this far along because now adding the rest of the necessary functionality is a breeze.

If for some reason you are totally unable to wait for the final release 0.3 and want to start playing around with it right now you can check it out on my darcs projects site.

Technorati Tags: perl, programming, flickr, flickr-tools

I’ve been slacking off on the Flickr-Tools project, mainly because it serves all my purposes as it is right now.

But then someone pinged me regarding the photos method in the Flickr::Person class (the one that will return all the photos for the user) and that was the incentive I needed to finally get to writing that method.

So now I remember why I left it half-way through…

The thing is that this method must use the flickr.photos.search method to get it’s info and it could return a lot of info.

So flickr.photos.search has a mechanism for managing the potentially huge volume of information —it forces you to break up your request into “pages” of results— which, of course, is the right way to do it.

But how do I map that onto my Flickr::Person class in a nice way? I surely don’t want to make the user have to control things like results per page and current page and so on, that would defeat the purpose of the whole thing, but I sure as hell can’t just go and fetch everything from Flickr because for a user with many photos I would blow up the computer’s memory fast!

So after thinking about it for something like 10 seconds I knew that the right way to do it would be to use an iterator. Instead of returning an array of Flickr::Photo objects I should return a Flickr::Photogroup object (or somesuch) and it should have a few simple methods like next, previous and total. You know, a classic iterator. When it ran out of results it should then invoque the Flickr::API::Photos::search method to get the next batch of photos.

OK, that is a bit more work than I bargained for but it is the right way to do it.

But then all of the other methods which I’ve already written and which return groups of things (mostly photos) should also return this kind of object. And each instance should know how to get more results when it needs it, so it should be rather smart about what it is representing.

This means two things:

So I will make this re-write and, unfortunately, I probably won’t be able to have it done by tomorrow as I told the poor soul who is expecting a new version of Flickr-Tools with the new method and, probably, no interface changes.

Should have though of that before, now shouldn’t I? Duh!

Technorati Tags: Coding, Perl

Since I know I’ll be off-line and unable to do much development on the next few weeks, I’m trying to get as much features as I can in there right now, so even small enhancements like the ones I did today are going to be pushed out to CPAN as a new version.

Release early, release often, they say…

For version 0.02 I cleaned up some documentation bugs (yes, I do take some care with the documentation) and added a couple of methods which where no brainers to implement but I hadn’t had the time to put in there.

There is at least one other method which would be really useful (get the list of all the photosets a given photo belongs to) but I’m waiting to see if the good folks at the Flickr API developers team can give us that method so as not to force me to do a lot of API calls just for this one. I do hope they find the time, but with all the buzz surrounding the new auth specs proposal I’m not that confident they will. First things first, I guess…

Anyway, let’s see how much more I can do before going away on vacation at the end of next week.

Technorati Tags: Coding, Flickr, Perl

Well, that’s just it, Flickr-Tools is up on CPAN and should be made available any time now.

I’ve decided that I just can’t sit on it any longer. It is far from complete but I’ve just looked at a big thread on the Flickr developer’s mailing list with the promising title of “Big changes to the Flickr API” and which I’m guessing has something to do with the authentication methods or somesuch.

So, given that I’m rather pressed for time as of late (contrary to what I’d expected, sigh…) I’ll just post it as-is, limited and all, and start work on version 0.02 ASAP.

As it is, it already allows people to get information on persons, photos and photosets and to manipulate high-level…ish objects with this info.

With the modules structure I’ve put in place it is really easy to add support for more core API methods so now it is mostly a question of filling it up with what’s missing.

Oh and I have to read the thread carefully to see how the authentication stuff will be handled (if that is what the thread is really about).

Getting this out the door is basically a way to get some testing for free on other platforms and to give anyone out there curious enough about this a chance to see my main design ideas for this distribution, so suggestions are always welcome. Help in the form of code is also greatly appreciated, but I’m not holding my breath… ;-)

Technorati Tags: Coding, Perl, Flickr

Following yesterday’s post and in the interest of getting something out as soon as possible, I’ve made some headway into the Flickr::Photo class today.

Of course this was after some cleanups and slight changes to the Flickr::Person class.

I can now instantiate a photo object, see (part of) it’s info and get an object representing it’s owner, all auto-magically. Rocks!

In the mean time, it is a good thing I decided to leave the write part of the read/write equation for later as I noticed today (two days late!) that there was an email on the Flickr developers list informing that the ‘setters’ (the write methods) where changed and that they now all have to be done via POST instead of GET.

While this makes perfect sense, I’m pretty sure it will warrant some changes in the Flickr::API and until Cal makes those changes I’ll keep the write part on the back-burner. (As if it weren’t already there…) ;-)

The hardest part of making something like Flickr-Tools is that you just know that the second you put it up on CPAN there will be a ton of changes to the object’s API which will become immediately clear.

Oh well, release often, release early, warn your users, listen to the community feed-back and all that jazz.

I really wish I could make the first release during the coming weekend… I’m not sure I’ll be able to do it though, as real life is getting in the way. But I’ll surely try.

(Did I just say “real life”? Hum… It seems my moo/mud-user personality is trying to take over when I’m not looking!)

Technorati Tags: Coding, Perl

I just couldn’t resist and after having finished the first version of the Flickr::API::* classes I just had to try and implement one of the high-level modules to test it out.

So I now have Flickr::Person full implemented (no tests just yet, though, I went way about this one the impatient way).

It took about 30 minutes to do it and works like a charm! It was extremely easy to write, it is lazy in getting data and, once it actually has to get any data, it caches it for future reference.

It’s great when things ‘just work’ like this. Quite a rush!

Well, but now I’ve got to be a good boy and go to bed, tomorrow I have a chiropractor’s appointment really early and I will pay for staying up late working on this.

Can’t wait to get the other three modules finished. This version of the modules is read-only (you can only get information from Flickr, and not all available information at that) but once I get it out the door and into CPAN I’ll start implementing the write part of it. And filling in the gaps on the read methods, of course.

Technorati Tags: Perl, Coding

Mock objects essentially rule.

With them I can trick my modules into asking my mock object for the data they would receive from a call to a third object (of which I know nothing about, except for the output it returns for a given input) and they are none the wiser about it. This is incredibly easy to do, and I’m not even sure I’m using it in the best and most efficient possible way!

So I’ve basically put aside the problems of testing with live data, with all that entails about having to have a live connection to the Internet, hopping that the Flickr API is responding, having to setup test data and so on.

Some people did make the point that this way, should the Flickr API change in the future, my module will still pass all the tests and then bomb out on the poor unsuspecting users. Which, of course, is absolutely true, but it is not what module testing is about. Module testing is about making sure that the code runs as expected on the platform it is being installed on. The API may change in the future, but this is something I have to find out by myself and make the necessary changes to the modules and release them again.

So now I’ve got a bunch of modules which take the results from the Flickr::API calls and turn them into a saner structure for a few selected calls (no time to do them all so far, but I’ve got the infrastructure set up so now it is just a question of filling up the gaps). All the calls I wanted to include in the first release of the distribution are done. Now to get the higher-level modules working with these and abstracting the User, Photo and Photoset concepts and off they go to CPAN.

This is actually fun!

Technorati Tags: Coding, Perl

The responses to my post yesterday about testing with Flickr, both at use Perl; and here at nowhereland were great.

I made the post rather in a hurry and right after I’ve come across the issue so I really hadn’t given it much thought.

Had I thought about it, I would surely have gotten to the solution that some people pointed to, which is that I should just setup an account for testing or even ask the Flickr crew to set one up for the community to use. And that would have been acceptable, even if I weren’t totally happy with it. It would always feel like a kludgy solution which could require tweaking if ever someone made changes to the user data or it’s photos (which people would be able to do given the info on the module’s testing scripts). It would also mean that you’d have to have connectivity when you where installing the module (or force-install it which I’m always loath to do).

In the end it is a good thing that I did make the post in such a rush manner because another solution came up which has a lot of promise.

Two people pointed me to the concept of “mock objects” which, as Melo puts it, “represents an ideal Flickr, which always gives me the response I need”. This seems just perfect for this situation.

So now I have a new goal: learn how to use _ Test::MockObject_ and integrate it into the Flickr-Tools development process.

Who would have though it? It actually pays off to be lazy! Oh wait, all geeks know that… Right…

Technorati Tags: Coding, Perl

I’ve been getting some good progress on the Flickr-Tools distro, mainly on the lower-level modules (Flickr::API::Photos, Flickr::API::People, etc). At this point I can already rewrite the pet project which got me into writing this group of Perl classes in the first place, using only these classes. Admitedly this is not much to boast about, but it feels nice anyway… :-).

On the way I’ve met some dead-ends and had to track back a few times, but nothing too alarming.

Only now I’ve come across a real stumper.

While developing the modules I’ve taken the “build your tests before writing any code” approach which is all fine and dandy while I’m working on it, but I just don’t know how I’m going to release the modules with the tests as they are now.

The problem is that testing anything useful at all requires not only the usage of a Flickr API key but also some rather precise information on a person’s info, a few test pictures, photosets, and so on.

Of course I’m using my own information for the tests, but I’m not really happy with releasing a module into the wild with all that information on it!

I know I could just capture the result of the original Flickr API response for each instance I needed it, change some values around and test my modules against that result but that is just evading the problem and a whole lot of extra work.

I just don’t know how I’m going to tackle this one…

Technorati Tags: Coding, Perl