Just another developer blog

Vim color scheme samples

2012-01-31T10:38:00.006+01:00

I have a lot of Vim colorschemes, and I like to change them very often, reflecting my mood. I needed a way to showcase them all and quickly pick one.

The original Vim Color Scheme Test script by maverick.woo is written in Perl and the build works on Windows systems. I wanted to add some new features, and to test it with my own colorschemes, but as I'm not very confident with Perl, I preferred to start over with a new Ruby version instead of forking his project. Here's my version (and here's the github page):

The script loads all your colorschemes from your default vim directory (~/.vim/colors), and writes into the output dir an HTML file for each colorscheme, with a render of a Ruby file using this colorscheme. It also writes a different copy for each language present in the samples/ directory. It also builds an index page for each language, with a showcase of how the colorschemes render the sample code, a download link for each colorscheme and a nice lightbox to preview it.

To make it run, you'll need:

ruby
macvim
tilt rubygem (to render the index template)

What still needs to be done:

Separate light and dark colorschemes
Make this work with versions of vim different from MacVim
Add the current language name to index pages
Add more languages (currently only Ruby and Python are supported)

ATM, the script uses a vim server named VIMCOLORS and sends it remote commands. This was made to make it faster, because opening a single macvim instance for each script required too much time. However, the --remote-send command of vim doesn't wait for previous remote-sends to be completed, so I had to add a "sleep 1" command in the script to prevent it from messing up the execution flow. Any hint to solve this is greatly appreciated.

Using virtual attributes for multi-parameter form helpers in Rails

2011-10-28T11:07:00.004+02:00

In a Rails application I am working on, I needed to setup a form with a field with a non-standard behaviour. The field represents a date object, so the FormHelper date_select helper looked great; however, the date to display was not the actual date to be set on the database, but the day before. Changing all the data on the DB was a bit risky, so I had to stick with this requirement.
I decided to use a virtual attribute to do this, as it seemed the most elegant solution, so I wrote this in my model:

And this in my controller:

However, trying to send this data to the controller resulted in a "1 error(s) on assignment of multiparameter attributes" error.

The solution I found after some search was this one:

Reference:
http://gabeodess.heroku.com/posts/14

OSX Lion: "This disk cannot be used to start up your computer"

2011-07-20T22:42:00.004+02:00

Today I downloaded the installer for OSX Lion from the App Store. I ran it, and when it came to choose the disk to install to, it told me that "this disk cannot be used to start up your computer". I had a setup with 4 partitions: the hidden EFI system partition, the Macintosh HD partition, and two linux partitions (an ext3 one + a swap one).

So, the solution was to resize the Macintosh HD partition. I booted my Mac from the Lion install DVD (you can burn it from the installer package, just search google to know how to do it), went into disk utility, checked all partitions for errors (this was necessary because I wasn't able to resize them otherwise) then I resized only the Macintosh HD partition shrinking it by some GBs.

After this, I ran the installer again and installation started smoothly.

HTH!

Machinist vs Factory Girl: Machinist win!

2011-06-19T10:28:00.006+02:00

Today I decided to verify if Machinist could be a good replacement for Factory Girl. In our project, we have a big problem with Factory Girl: even if you tell her not to hit the database, using the Factory.build method, if an object has associations, these are saved on the DB. And this causes a huge slowdown in specs using factories. We've been using Factory Girl for nearly two years, and if we could find a way to stop him hitting the DB, we could really have a huge improvent in our test suite running time.

To verify if Machinist could perform better, I set up a basic rails app. Look at this example:

If you do a tail -f log/test.log and you run this spec, you'll see something like this:

The Factory.build method has to save dependencies on the DB to set the foreign keys on the objects and validate them.

Let's try with machinist:

This time, running tail on the test.log file and running the spec, doesn't shown any DB hit, and of yeah, we have a green test.

I verified this also by putting a debugger line after the validation and inspecting the DB from within the debugger after the validation has run - with FactoryGirl, it revealed an Address object saved on the DB, while with Mechanist it didn't.

I still haven't looked inside machinist to show how it handles this, but I'll do it soon, so keep in touch!

Howto: Run a rake task in sandbox mode

2011-03-10T14:51:00.003+01:00

If you have a Rails rake task that somehow changes your DB data, but you want to be sure that the DB will be rolled back to its previous state after the rake task has completed, you can simply include this snippet right after your task definition:

If you wonder where is this code coming from, it's directly from the rails console code.

Rails 3 scopes with HABTM (has and belongs to many) relations

2011-01-28T14:34:00.004+01:00

There are already many posts about this, but maybe this simple example will help you understand this subject even better.

invalid option: --with-pg-dir=/opt/PostgreSQL/9.0

2011-01-18T16:03:00.001+01:00

I'd bet a lot of ruby devs actually found themselves stuck in this problem. You checkout a github repo, you run a bundle install and = duh = a gem cannot install because of a missing library.
You're sure you've already installed the library or dependency or whatever, but in a different path from the standard one (in this example I'm talking about PostgreSQL installed via the graphical installer instead of the ubuntu apt repo); so you issue the command

gem install pg -v0.9.0 --with-pg-dir=/opt/PostgreSQL/9.0

And you get this error message: invalid option: --with-pg-dir=/opt/PostgreSQL/9.0

What's the problem? You need to separate options with another pair of dashes:

gem install pg -v0.9.0 -- --with-pg-dir=/opt/PostgreSQL/9.0

And everything will work.

Format the results of a MySQL query like MySQL! In Rails!

2010-09-30T12:50:00.013+02:00

It may happen that you need to display the results of a MySQL query on a page. E.g., your customer asks you to add a report on a page, and you don't want to build a custom template, but just write the query and see the results.

We can do this easily thanks to the terminal-table gem (see http://github.com/visionmedia/terminal-table). This gem allows printing an ASCII table, just like the one you see when you use MySQL from the terminal. Look at its page on GitHub to see how easy it is.

To integrate it with MySQL and Rails, we can use ActiveRecord::Base.connection.execute("some_sql_query"). This method extracts the result of our query to a Mysql::Result object, which consists of a set of hashes with the results of the query. We can navigate through this hashes iterating over the all_hashes method, and throw these results into a table. Here's the code:

So all we need to do is include our module and call the method print_results_of_query. Look at this example in script/console:

All you have to do if you want to put this in a page is wrap it into <% and %> markers in your .html.erb template. Have fun!

SCREENCAST REVIEW - Vim for Rails Developers, by Ben Orenstein

2010-07-02T08:18:00.001+02:00

(link: www.codeulatescreencasts.com)

In the last months I've been working on a Rails project in an Agile team. I worked mostly on TextMate, and its speed is really amazing. It has a full set of features, with snippets, bundles, syntaxes and so on. It has many shortcuts, and apparently there's no need to switch away from TextMate.

But, it has its drawbacks: it's a Mac only application, so I can't use it on my Linux box. Also, it's a commercial application, and even if its cost is not too high, I don't like to pay for software. Finally, it's a GUI application, and it cannot be used over SSH to work on a production machine.

VIM always looked like the perfect answer to these needs - but yet, getting the productivity I reached after one full year of TextMate requires some time. Every time I tried using VIM for some serious work, I ended up discouraged, because even the most basic stuff like file navigation and launching tests took ages, compared to the snap of fingers of TextMate.

I always like to challenge me though, so when I saw this screencast by Ben Orenstein I immediately decided to give it a try. And yes, it was a good decision!

The screencast begins with some general hints about typing speed, keyboard layouts and Dvorak keyboards. Even if it may not look strictly related to the main subject of the screencast, I appreciated this part, and I think many people will find it useful.

Then the video moves on to the vim-rails plugin by Tim Pope and highlights its main functionalities. Here you'll see clearly how this screencast is mostly intended to tell "stuff that matters" instead of just giving a plain list of features that you'll never use in ages.

In this section, the screencast shows how to launch tests, navigate between files, and execute Rake commands directly from your VIM session. I think this plugin has really a lot of features, too many to be covered in this screencast; Ben managed to make a good choice, selecting the fundamentals you'll use everyday.

Another amazing plugin is Snipmate: it allows to insert snippets of code by just inserting a few characters and hitting TAB. Ben quickly overviews it and shows its use - another must-have for TextMate aficionados like me ;)

After this follows a section about ctags and their integration with Rails and VIM. This is also really useful, and combined with rails-vim it provides a quick way to navigate between project files.

Finally, there's an overview of ACK - a grep replacement, focused on ease of use and speed - and its simple VIM integration. I'm also using Ack.mate, so I already knew it, and it's one of those tools you feel the lack of when you don't find it installed.

Before ending the screencast, Ben also shows an overviews of single commands and configurations that he finds particularly useful. Again, they're being selected with usefulness in mind, so you'll end up with that "Wow, I need to try these now!" wonderful sensation.

What else can I say? The screencast is well done, has a nice music, a simple and clean layout and it's spoken in a clear English ;) This should be obvious but I've found a lot of screencasts with crazily fast voices, so it's not so obvious :)

To improve this screencast, I have nothing to say about its content: it's really great! I would only add some OSD with keys when key combinations are being explained, so that it could be even simpler to memorize them.

I would definitely recommend this to any Rails developer wanting to try VIM, or to any VIM user who is starting to code Rails and wants to boost his productivity. It has a good amount of tricks and hints that can be useful both for the VIM neophyte and the VIM master starting a Rails project.

TextMate Syntax Highlighting Howto: A simple todo list

2010-05-14T14:14:00.009+02:00

Today I wanted to add a syntax highlight for my todo list favourite format to TextMate.

Here's how to do it. In TextMate, go to Bundles, then Bundles Editor, then Edit Languages...

Click on the plus button in the lower left corner and choose "New Language". Paste the following code in place of the example code provided.

This defines three patterns for each condition. They should be self-explanatory, I used only simple regexps here. Save your language definition, and it should appear in the languages combo of your Textmate.

To complete syntax highlighting, you also have to add the colors to your current textmate theme. In the application menu, go to Textmate, then Preferences, then Fonts & Colors. For each pattern name, click the plus button and create a new element. It must have the pattern name as scope selector. Choose colors as you like.

References:

Textmate help
Mac Dev Center

Ruby Mixin and monkey patching examples

2010-04-23T08:23:00.008+02:00

Let's explore a couple of solutions to dynamically add a split_by_half behaviour to an array object. The first technique is the mixin: it allows to add the method to a single array instance. The second one is called monkey patching, and adds the method directly to the Array class, adding this behaviour to every array instance.

Script to download the latest Chromium build for OSX

2010-04-12T09:08:00.005+02:00

The way we plan

2010-02-22T14:51:00.002+01:00

In our Agile team, we have iterations lasting one week, and we plan our work every Monday morning. Our customer comes to our office, we gather around a table, and we watch the project backlog to choose the new stories to be worked in the current week.

For the majority of the User Stories, we already have an estimate made at the beginning of the project. This estimate has been made in Story Points, and sometimes it's really inaccurate, because when we started our project some features looked really different to our eyes; so, we have to re-estimate these stories. For this we use another measuring unit, the Pomodoro (look here if you don't know what I'm talking about).

After these estimates, we decide with our customer which user stories are to be worked in the current iteration, trying to balance the business value and the stories costs. We plan stories to fit our work capacity for the iteration (e.g., 3 developer pairs work each 10 pomodori per day, so in a week we can plan 150 pomodori/pair).

Of course, when we discuss features with the customers, some new stories may be introduced, some could get splitted, and some others delayed for future releases.

There's something wrong with this. First of all, estimating in Pomodori needs a lot of deep analysis to be made to get to an accurate estimate. This slows the estimating process a lot, because we end up talking about implementation details. Of course our customer gets bored soon, as he's not a technical customer. We introduced this way of estimating after the first iterations of the project, because we were estimating using Story Points, and we were having a very low accuracy; later, we never tried to switch back to Story Points, even if now our accuracy could have improved. Just to mess things up, we also use a "fake" Story Point measuring, obtained just multiplying pomodori by 10. So we estimate 10 pomodori, and we write 1 story point. Using the Real Story Points could speed up our planning work a lot, because we could step up by an order of magnitude, and estimate user stories comparing them with the ones already worked.

Another problem we are having is that we're splitting stories too much. We're following a rule of thumb, like "no stories bigger than 2 points". Such a rule had been voted in a retrospective, after a couple of weekly fails. We had some big stories (like 4 points stories) not being accepted by our customer because of minor issues; so we decided to split them up, to minimize the impact of a rejected user story on the iteration score. The wrong thing with this approach is that we're just changing the way of measuring our system to obtain better measures. Think about it: an iteration ends, and you discover a bug during the demo. Of course you'll have to fix it in the following iteration, and of course this will cost you some additional work. If the iteration scores are 2-10 or 5-7, nothing changes in the system; you still would need to do some additional work. The only thing changing is a couple of numbers written on a spreadsheet. Also, different iteration scores would impact only on the short term velocity, and would instead have no effect on the long term velocity. The short term measure gives no confidence and is much less important than the long term one, so there's no need of getting better short term results. This "hack" on measuring, in our case, is also introducing new problems. We spend time thinking about ways to correctly split stories, we introduce unneeded dependencies and we get to stories which are really difficult to demonstrate to the customer.

Some ActiveRecord fun

2009-09-03T09:56:00.006+02:00

Many of you have already used has_many and belongs_to Active Record relationships. This is really easy to do when you have a single attribute to map; like, in example, an Article with many Comments. You’d write something like

class Article < ActiveRecord::Base
  has_many :comments
  ...
end

class Comment < ActiveRecord::Base
  belongs_to :article
  ...
end

Combining these declarations with a article_id column in your comments table would allow you to:

access all the comments of an article with article.comments

access the article a comment belongs to with comment.article

There are situations, however, when things get a little more tricky. Think about this example: you have a Movement object, which identifies a cash flow between two Accounts. You also have an accounts table; so, in your movements table, you would add columns like from_account_id and to_account_id.

How can this be mapped with ActiveRecord? We need to write two belongs_to declaration, this way:

class Movement < ActiveRecord::Base
  belongs_to :from_account, :class_name => "Account", :foreign_key => "from_account_id"
  belongs_to :to_account,   :class_name => "Account", :foreign_key => "to_account_id"
  ...
end

This tells ActiveRecord that the movement class must reference two Account objects. One is accessed with movement.from_account and the other with movement.to_account.

You can do the same thing for your Account class:

class Account < ActiveRecord::Base
  has_many :movements_out, :class_name => "Movement", :foreign_key => "from_account_id"
  has_many :movements_in, :class_name  => "Movement", :foreign_key => "to_account_id"
  ...
end

This way, with account.movements_in, you can get all the cash flows to the account, and with account.movements_out all the cash flows going out of the account.

Plain and simple! :)

Autospec for Rails + libnotify on ubuntu howto

2009-08-15T14:33:00.005+02:00

Hallo! After a couple of hours of work I finally managed to get autospec popup notifications on my ubuntu machine. Now I can start autotest in the background, and whenever I save a file into my project folder, after a few seconds I receive a notification message on my desktop about the tests.
Here are the steps to get the same result:

install the required gems (ZenTest, and obviously rspec and rspec-rails);
sudo aptitude install libnotify-bin;
edit the file .autotest in your home folder (I said home, not the root of your project!) with the content from this link: https://gist.github.com/840314
save the two pictures rails_ok.png and rail_fail.png from http://blog.internautdesign.com/2006/11/12/autotest-growl-goodness and put them into your ~/Pictures/rails folder.

At this point you are supposed to only run autospec in your project root folder, save a file and see a notification like the one at the top of this article! Happy BDD!

Building a pronounceable password generator in Ruby

2009-08-09T15:44:00.003+02:00

Heya guys! Today I tried to build a pronounceable password generator. To keep it simple, I started from a simple concatenation of words - in my case, italian words. This is really easy, but you'll see that results will be very good.

I took a list of italian words from this site: Italian dictionary and affix file for ispell. It's a good choice because ispell dictionaries are prepared to be declined; concatenating two words without suffixes will give good results for pronounceability.

Before proceeding, however, I needed to narrow the words list:

I removed the ispell affix definitions, to get words like "abaco" instead of "abaco/G":

cat italian.words | sed 's/\/[A-Z]*//g' > parole.txt

I removed all words containing capital letters, like "Yamaha", "Windows", and "Acicastello":

cat parole.txt | sed '/[a-z]*[A-Z][a-z]*/d' > paroleNoCapital.txt

Finally, I sedded out some cuss words - not writing them here, for the sake of decency! :)

I was then ready to write the following ruby script:

def genera_pass_da(parole, n = 2)
how_many_words = parole.size
word = ''
1.upto(n) do
word += parole[rand(how_many_words)].chomp
end
word
end

file = File.open("parole.txt")
parole = file.lines.to_a.select { |word| word.size < 7 }
1.upto(10) do
puts genera_pass_da parole
end

Even if it's far from being polished and optimized, launching this gives a good set of pronounceable passwords. They're all lowercase and without numbers - so probably not the best choice for your remote banking account - but still ok for other uses. Here's a shot:

metalelf0@eagleone$ ruby rapg.rb
uditovoi
vammimidi
gechitua
dareisisma
ecoivi
ancabatto
unotacca
fangopupe
apesella
bemasire

Someday I'll improve it by adding numbers, some capital letters, customizing choices and so on - but many friends are yelling me that it's sunday and I need to go out, so... see ya soon!

Shared pomodoro part II

2009-06-28T10:44:00.004+02:00

Last month I posted an article about the Shared Pomodoro, a shared timer for all the members of an Agile team. By sharing a common timer, all the team members can have pauses at the same time; so, there's nobody hanging around speaking or joking when someone else is working.

We are trying to apply this in our Agile team, without much success. The main area of debate is focused on pauses: a lot of times the developers keep working for 1-2 minutes after the ring of the timer, and then they want to take a 5 minutes break like their team partners. But in a shared timer environment, this is not possible. Shared pomodoro should also mean shared pauses.

Also, when somebody is on a difficult task at the end of a pomodoro, and wants to finish it before having a pause, it's common that he needs more focus and concentration. This is really difficult if the other developers are pausing, and talking loudly in the same room.

Another problem we had with pauses was that they sometimes extended after 5 minutes.
In our latest retrospective, a week ago, we decided to ban the usage of personal laptops during the 5-minutes break, to remove on of the most common sources of oversized pauses. However, some of the pauses lasted more anyway, because somebody was going out of the room to get a snack, smoke a cigarette, go to the toilet and so on.

Personally, I don't think that forcing all the team members to finish exactly at the same time and start over exactly at the same time could be more productive than leaving each couple free to take its own rhythm. If a couple works 28 minutes and then takes a 5 minutes pause, they're not working less than a couple doing 25-5. But I think that the couple working 28 minutes still needs a 5 minutes pause, and it wouldn't be correct to force them to a 2 minutes pause just to keep the shared pomodoro rhythm alive. I know that the strict application of the Pomodoro Technique would prohibit any work after 25 minutes, but in my experience I think this doesn't compromise the technique effectiveness too much. After all, the technique leaves space for flexibility in the pauses duration (both short and long pauses), so even applying it strictly could lead to not synchronized couples.

I've read on the Pomodoro Technique official mailing list (http://tinyurl.com/l55un2) that some other teams tried using a shared pomodoro without much success. At this point, I think the best solution is having a pomodoro for each couple. It has its downsides too, but IMHO they can be overcome much more easily.

Don't leave juniors on the bench

2009-05-21T09:29:00.005+02:00

Recently I've read many articles about how to choose the best junior developers, and how to train them. This is an hot topic for project managers, as they need to have good and cheap developers: usually, this also means "young" developers.

I still haven't found any article, though, approaching this subject from a psychological point of view. Typically, these writings are about technical training, or maybe about how to productively insert a junior resource into a team: but not about how to make the junior motivated to give everything for the project.

I think it's a really different situation from inserting an already expert developer into a team. A young developer still needs to understand almost everything about team work, daily planning, and so on. Sometimes universities don't give them any hint about the situation they'll find in a real work place, so it's time for them to find out. I suggest, let them find out the hard way.

A young developer needs to be involved in all the team activities. He really should be able to write a lot of code, discuss planning, design and so on, since the start of his work in the project. In an agile team, this means that he should write code at least as often as he drives his pair; he should be able to speak while the team writes down user stories, and participate actively to retrospectives.

Don't care too much about the impact this could have on both the developer and your team: in the worst case, you'll have a small loss of productivity, but we'll discuss this in detail later in this article.

I think a good analogy can be done with football. Yeah, I wrote football - after all, it's a team game, where the cohesion between all the team members is important to achieve success.

In a football team, as players get old, young players come in, and they need to be inserted into the first team. This process can be really difficult, no matter how good the young players are: for a coach, leaving the 33 years old world-class striker on the bench and throwing the 17 years old promise on the field, could result in both a success or a failure covered with critics.

However, if you ask any football coach what's the best way for young players to improve, they will answer the same: experience. They need to *play*. A young player, if left on the bench, can not grow, even if he has the chance to train with the best footballers in the world. He needs to feel the adrenaline of important matches, receive the ball and face the toughest opponents.

Going back to our junior developer, it's exactly the same situation. If you want him to improve, you must not leave him on the bench, thinking that time will help him.

Such a cautious and mild approach could result in a demotivated young resource, compromising both his morale and his productivity. Also, you'll need a lot of time to evaluate his real skills.

You should let him code, even if he's half as fast as your other developers, and hear his opinion about the practices of your team. Involve him in all the team activities, and try to teach him what he needs to start working with your team.
Obviously, this can be done only if he has at least some basic skills, but choosing the best young resources is another topic - I won't dwelve into this right now.

Maybe you'll realize he's such a good developer that after some weeks he'll be perfectly inserted in the team, contributing interesting ideas and giving a productivity boost, but even if this takes more time, it's the right thing to do.

Such a steep approach is obviously not productive in the first weeks, but as time goes on, you'll be able to evaluate better not only the technical skills of your junior, but also his resistance to pressure and work load. These characteristics are as important as the technical skills, maybe even more, so finding them out is definitely useful.

Shared pomodoro - to share or not to share?

2009-05-11T11:04:00.003+02:00

Last week a very interesting discussion emerged in our Agile Team. We are 10 developers, and we all work in the same room. We use the popular "Pomodoro Technique", conceived by Francesco Cirillo, to organize our daily work. If you have never heard about it, visit www.pomodorotechnique.com; shortly, its main characteristic is that it divides the daily work into very small iterations called pomodori, all of 25 minutes, separated by regular pauses of 5 minutes.

The technique works really well; we had some discussions about the nature of timers - somebody loved the kitchen mechanical ones, somebody else was irritated by their ringing sound and preferred software timers. But apart from this, there were no problems.

A month ago, we added two couples of developers to our team. The new guys came from another popular Agile team, and they proposed the idea of a "shared pomodoro": an unique timer being shared by the whole team. The Pomodoro Technique defines clearly how to setup work iterations and pauses, and it's also really strict about the need of stopping work exactly after 25 minutes; so, there's no reason for not synchronizing all the developers.

Before this fact, we didn't follow the technique too strictly; somebody prefered to work for 50 minutes and take a longer pause thereafter; others used pomodori just to track their work, without caring too much about pauses duration.

As you can imagine, the shared pomodoro is quite difficult to apply in such an environment.

Let's try to summarize briefly the advantages of a shared pomodoro:

- pauses are shared too, so the whole team can have a pause together;
- when you're working, everybody else is working too; so, you don't have distractions from other people having a pause;
- there's more discipline: all the developers have to keep the same rhythm, even if they're tired;
- the team actually behaves like a team, and this common practices may increase the cohesion between the team members.

The disadvantages of the shared pomodoro are almost symmetric:

- if you *really* need to take a pause, for any reason, you break the team rhythm;
- if you want to work for more than 25 minutes without a pause, you can't;
- every time there's a pause, the whole team has to wait for all the team members coming back to the open space (but hey, this is called discipline!);
- the team behavior looks more chaotic, and it's more difficult to organize common activities.

So it's mostly a choice between discipline and relaxed rules. It's not easy to choose which solution is the best for you, or which one will give you more productivity. The author of the pomodoro technique, Francesco Cirillo, reports that studies have been made to find the maximum time a human brain can keep concentrated on a single subject, and the average value found out is 25 minutes. So, if his theory is right, following the technique strictly is the best way to keep concentrated, work better and consequentially improve productivity.

Enough said? Not exactly. In the previous statement, there's an important word: "average". This means that the "25 minutes" value is not the same for everyone, or for any situation. Sometimes it may happen that you work on a really hard subject, and after a pomodoro of work you need a slightly longer pause, because you're tired; some other times, you're feeling a good rhythm, with many tests passing, and you want to take a shorter pause. This kind of behaviour is definitely not compatible with a shared pomodoro.

If you want to decide whether a shared pomodoro is right for your team, try to ask your team members how confident they are with the rhythm described by Francesco Cirillo. If they're already using the technique strictly and they feel confident with it, you can try out the shared pomodoro for a month or so, and then decide whether yo keep it or not. Otherwise, you can try to tend to it, if you feel that your team is too chaotic.

The only approach I find not correct is imposing a shared pomodoro from above, without asking any opinion to the team members. This may lead to unhappy team members, cause they could feel their freedom and creativity is constrained by a timer. If all of your team members feel like this, maybe you should put the shared pomodoro apart - or find other team members! :)

MySQL - CouchDB performance comparison

2008-09-25T11:35:00.003+02:00

Here you are with the results of our benchmarks!

CouchDB performance testing: first results

2008-09-24T12:47:00.003+02:00

We are trying to test the performances of CouchDB by measuring the times it takes to execute a query (=a view) in different conditions.

We used a modified version of the default test suite - I'm not posting it now cause the code isn't very readable, but if you need it, just ask and I'll fix it up.

In this version we put some functions to create a named view and execute it with a single argument; then we ran the test suite from the Futon utility, we collected execution times from the test suite and put them into a google document.

Results show more shadows than lights: the database is fast when executing queries already indexed, but it's really slow into creating indexes.

We are working to expand this test with different sample-data size. So keep in touch!

2008-09-24T11:14:00.002+02:00

Here is a mind map I made to explain the differences between CouchDB views. Thanks to FreeMind application!

Yesterday... and plans for today

2008-09-24T08:42:00.003+02:00

Yesterday we completed two user stories of our project in java using couchdb4j:

- add a custom field and be able to see it in listings;
- find records by a specific field.

Then we switched to a more difficult task: evaluate the performance differences between CouchDB and MySQL. It's a difficult task because the two databases are logically very different, and it's not easy to find the common operations to test and compare.

In example, the "find" operation in MySQL can be done with a query; it can be a compiled query or an interpreted one.
In CouchDB we need to create a view function and execute it; we can have "named" views - that resemble the MySQL compiled queries - and temporary views, that are executed on demand.

In the CouchDB Futon utility - the web application builtin with CouchDB - it is possible to run a suite of tests and see their execution time.
Editing the file /usr/local/share/couchdb/www/script/couch_tests.js it is possible to modify the default tests or to add new ones, so we quickly wrote a new javascript function to check the execution time of a named view.

Our results were really poor for the first access, when the view is created - around 1 second to create a view on a single field, for a database with only 5000 records; after the view was created, access times were much slower, in the range of 5 to 15 ms.

I don't know if this problem is related to any error we made in the test-writing process, but I'm pretty sure I can exclude this; maybe there is another way to iprove performances of the standard javascript view server, I'll try to find out today. Otherwise, we'll have to change our project, because such a poor performance would make CouchDB not usable in any productive environment.

Keep in touch! Andrea

What a day of work! :)

2008-09-22T20:03:00.003+02:00

Today's work has been really intense. You can see this also from this post's issue hour: it's 20 pm here, and for the first time I took some work home.

Today we started working on the CouchDb handling of custom fields. In our application - you can see the link in the previous post - we already wrote the CRD parts of the CRUD; we needed the Update functionalities.
At first we wrote our code keeping the previous structure with fixed fields in the class Person. This was not so good for the CouchDB approach: when we had to insert a custom field, we had to overcome the limitation of the fixed fields.
So we decided to make a major design change into our application: we switched from the standard class design to a more flexible one, where instead of fixed named attributes we put them into an HashMap.
This way we could handle variable-length lists of attributes, and also attributes with variable names; in a while it was possible to handle the update part successfully.
We had some problems because a part of our application was still old-fashion code, and we forgot to convert it; when we got that, we felt really stupid, because we were getting mad looking for the cause of some missing functionalities.

Today's work

2008-09-19T17:40:00.002+02:00

Today we worked on the project addressbookcouchdb4j. It's an explanation project for the couchdb4j libraries, and I hope it will prove itself useful for anybody trying to work on couchdb in Java.

I found some difficulties in this work. Some were due to my not perfect knowledge of java; some others to the lack of documentation on both couchdb and the libraries. However I contacted Marcus Breeze, the developer of couchdb4j, and he explained us something we missed. Maybe we could join our works to help the couchdb user base: thanks a lot Marcus!

The current state of our work can be seen easily at the following address:

http://www.bitbucket.org/metalelf0/addressbookcouchdb4j/overview/

Feel free to clone the hg repository if you want to sneak at the code ;)

Actually our code allows to insert, remove and list elements into a simple couchdb database. It's an addressbook - jeje, not so original, but it was the simplest test case that came to our mind. We'll add some more functions in the next days, so stay in touch!

What we did in the last two days

2008-09-18T08:55:00.002+02:00

This is a summary of the work I did with my colleague, Simone Albertini, in the last two days.

Tuesday, september 16

Study of couchDB on the official site. Read the introduction, technical overview and wiki. Studied the difference between relational and non-relational DBs, and concepts of document, view.
Local installation on a linux VM of a working CouchDB.
Work on the local DB. Experimented the http communication system with CouchDB. Tried the ruby basic access library.
Started using CouchDB in a java environment. Installation and tryout of the couchdb4j libraries and needed dependencies.

Wednesday, September 17

Getting deep into CouchDB.
Brainstorming about possible techniques we can use to reimplement a relational DB into a document-oriented one.
Downloaded and studied the source files of couchdb4j, ran the tests included, and saw they didn't pass.
Partial reimplementation of the provided tests, to make them pass and to understand the library behaviour.
Read some introductions about Lucene and its applicability fields.

Relational to document based: what to do?

2008-09-17T11:43:00.005+02:00

I'm trying to figure out how to "convert" a relational database to a document based one. Specifically, an existent (legacy) relational database.

As an example, think about a DB where the record can be represented by the following Java class:

public class Person {
String name;
String surname;
Date registration_date;
}

If I want to write an instance of Person to a document based DB, as CoachDB, I can use the following approaches.

The simplest one that may come to mind is making Person implement the Serializable interface, serialize the instance of the class, and then write the obtained bit-code to a Document. However, this would make the DB completely useless: in the DB I would have only bit-codes, and I wouldn't be able to retrieve data without the Java serializer/deserializer. I would use the DB as a file system, so this approach is definitely bad.

Another idea could be: for each field, write it to the document in a form like

{ field.key , field.value.toString() }

However this would work without problems only for the first two fields. In fact, when we write to the document the registration_date.toString(), we would have some problems reconstructing it back from the document.

I'm currently searching for a solution about this topic. Stay in touch!

CouchDB accessing issues

2008-09-17T10:22:00.004+02:00

I installed on my local system the CouchDB database, without any problem. Then I tried the given Ruby code to access the DB. It's really easy as CouchDB has no query language, but it uses http messages as directives; so, as I supposed, the Ruby code worked flawlessly.

THEN, I tried the Java code. I'm still trying it.Unluckily, as you can see on this page,

http://couchdb4j.googlecode.com/svn/trunk/javadoc/index.html

the documentation is lacking. I'll work on this during the next days to make it better.

CouchDB

2008-09-16T11:15:00.003+02:00

CouchDB is a document-based database developed by Apache Foundation. Its primary features are eventual consistency, high availability and extreme scalability. It's written in Erlang. Here's the related link on wikipedia: http://en.wikipedia.org/wiki/CouchDB.

Here's a link to a good installation guide on Ubuntu Hardy 8.04:

http://barkingiguana.com/2008/06/28/installing-couchdb-080-on-ubuntu-804

It works perfectly also with the latest version of CouchDB (currently 0.8.1).

15/09/2008: Erlang

2008-09-15T12:29:00.003+02:00

Today I'm continuing my Friday work on Erlang. Sorry for not posting about that, but we had some network problems and I forgot to update during the weekend.

Here is some sample code to explain TAIL RECURSION in Erlang.


-module(sum).
-export([

  sum/1,
  sum_acc_caller/1,
  sum_acc/2

  ]).

sum([H|T])  -> H + sum(T);

sum([])  -> 0.

sum_acc([H|T], X) -> sum_acc(T, X+H);

sum_acc([], X)  -> X.

sum_acc_caller(H) -> sum_acc(H, 0).

The basic problem we face is the sum of the integers in a list. In this code you can see two functions:

- the first one is sum/1. This is the basic, non-tail-recursive function. It calls itself (n-1) times, where n is the size of the list, and then returns the expected value.
- the second one is sum_acc/2. This is the tail-recursive function: it uses an accumulator variable X to keep the value of the sum while traversing the list. It keeps adding to this variable until the remaining part of the list (tail) has size 0; then it returns the sum.

Using JIRA in an agile environment

2008-09-11T17:01:00.006+02:00

Using JIRA in an agile environment

Although JIRA is not explicitely designed for use in an agile environment, its flexibility allows agile developers to adapt it to their needs, making it a good complementary software for project management.

After a first look, there are some obstacles in JIRA issue management, that we’ll need to workaround:

there is no “difficulty point” variable for the issues;
there’s no “iteration” concept: we’ll need to use the “version” variable to represent the iteration;
the available reports and graphs are based on the concept of tasks per version. However, in an agile environment, this information is not important, as it’s much more important the difficulty of the task than the number of them.

Finding a solution to these problems is not easy. Anyway, there are some plugins - both commercial and freeware - that can help us to improve our Jira experience. Let’s see the most important ones.

GreenHopper

GreenHopper by GreenPepper Software is at the moment the most used plugin for Agile developers using JIRA. It’s a stable project with very frequent updates (they claim one release every 2.5 weeks). But it’s not a free plugin: the licence prices range from 350 canadian dollars (234 €) to 1150 canadian dollars (768 €) per production server. They offer a free licence too, but only for managers of open-source projects.

GreenHopper is a full implementation of the Agile planning wall in a JIRA plugin.

It represents all of your JIRA issues as an index card; different types of issues are different colors. Each card displays the Summary and the other information crucial to planning, like estimates and time-remaining. It hides all of the other information: full description, comments, etc., but they’re all just a click away. You can drag and re-order the tasks. You can edit them on the fly, to add comments or log work.

The Planning Board helps you schedule many issues over an entire release cycle. GreenHopper also offers a Task Board to show you the work-queue for a version or component. And there is a Chart Board which will show you the burn-down chart for a release.

The chart board offers the following graphs:

Display of the burndown curve
Display of the team effort curve
Display of the estimation accuracy curve
Burndown chart based on a custom field
Burnup chart based on a custom field
Value chart based on a custom field
Issue filtering
Configurable start date and end date

The most evident downsides of this plugin are that the graphs it produces are limited to a single iteration (or version); and these graphs can be seen only after the iteration is closed.

This problem is mostly due to the internal Jira structure; like previously said, the Jira application lacks an “iteration” concept. The team effort curve for a single iteration, telling how many story points are being completed every day, doesn’t give any information about the overall project.

The Planning board, instead, is a good way to keep track of the story cards of the project. However many agile developers still prefer to have physical papers, sticky notes and a real blackboard.

Link: GreenHopper plugin

CFO Approval Required

CFO Approval Required is a plugin developed by Go2Group. Its original target was different from the one an agile group could use it for, but sometimes it may find its space. It’s a free plugin, so try it if you need it. It’s not maintained since the original 1.0 version, released in April 2007.

This plug-in is designed to allow a CFO or manager to approve issues based on the amount of a requested purchase. The issue is then forwarded for approval to a second user, such as a Controller or Project Manager.

This plug-in is also used for any generic approval process such as travel request, vacation request, Feature Request, Test Requests, PO, etc.

This plug-in can be useful in scenarios such as developer changeset (set of associated changed source code files) for verification to a team member, if approved by the team member the changeset is then marked as ready for test. This scenario has proven useful to shops working in xtreme - scrum - agile environments.

In my opinion, this plugin can be useful only in big teams, or in outsourced development teams. Agile practices encourage communication as the most important value, so decisions are usually taken in the open space. However in different environments this plugin could help a lot.

Link: CFO Approval Required plugin

Custom Issue Order

This freeware plugin is very simple, yet it can be very useful. It’s not being updated since November 2007.

(it adds a) Custom field to order issues. With this plugin, any issue list can be ordered in a custom way. Useful for work queues or fine grained prioritizing.

It can be a cheap alternative (even if less eye-pleasing) to the GreenHopper planning board: if your only need is to sort user stories, and you don’t mind colors and stuff, this one is for you. The custom field, however, cannot solve the big problems noted before: it’s related to the single issue, not to iterations (= versions).

Link: Custom Issue Order

Laughing Panda JIRA Agile Report Plugins

This free plugin offers at the moment a single graph: specifically, it’s a workload burndown chart (in hour or from a custom field like story points). Again, it suffers the same problems of its bigger brother GreenHopper: graphs are related to the single iteration, so they don’t give any “wide view” of the project.

Link: Laughing Panda

Agile Velocity Tracking plugin

This free plugin offers a report that will generate velocity tracking charts over versions. The report displays charts for Velocity Points tracking. Versions are treated as iterations. The points are gathered from a custom field which must be manually added. Iterations can be of variable length. Forecasting is currently done on ‘yesterday’s weather.

This plugin offers something really different from the previous ones: it’s giving the chance to analyze more than a single iteration. The information gathered from its graphs are usually more important to the agile developer’s eyes: points over stories, points burndown.

We could argue that this plugin needs a couple of iterations completed before proving its strength, as it doesn’t give any information about the current iteration; the predictions made by this plugin are not based on a very complex model, so they could be inaccurate and should be treated carefully.

The development of this plugin is not going on since August 2007.

Link: Agile Velocity Tracking Plugin

Agile Wall Plugin

This is another free plugin, offering a display of a Project Wall.

Report for viewing issue statuses in the same way as in agile project progress walls (issues in three columns: to do/in progress/done). Many agile teams use task walls to represent the current status of the sprint (or project). Tasks are grouped by their status to three columns: Not started, In progress and Done. Tasks are usually also sorted so that top priority issues are on the top and low level issues are on the bottom.

Agile Wall Report plugin tries to mimic this view in JIRA so that team members can use a same kind of a view for a current project version that is being developed.

Note that current version uses default JIRA statuses. Issues with “In Progress” status are rendered in the middle and issues with “Closed” status are rendered as done. Other status types are rendered in the Not started column.

The biggest limitation of this plugin is the impossibility of using custom issue status (e.g. “waiting for approval”, test related statuses and so on). However it’s still a good plugin, allowing a visual representation of a chosen sprint (or version) to be seen in Jira. Like said for the GreenHopper plugin, many agile developers prefer a real board, so the real need of this plugin is subjective; but this one is free, so you can try it and decide by yourself.

The latest release was made on 2007, July 17.

Link: Agile Wall Plugin

11/9/2008

2008-09-11T10:25:00.012+02:00

Morning task: try to figure out how i18n works.

Setting a text variable in a .properties file and referencing it in the view is working.

I.e., if the file is called assigned_portlet.properties, and its path in the plugin dir is

src/etc/com/atlassian/jira/plugin/portlets/example/assigned/assigned_portlet.properties

it should be referenced in the atlassian-plugin.xml with a line like this one:

<resource type="i18n" name="i18n" location="com.atlassian.jira.plugin.portlets.
example.assigned.assigned_portlet">

Now, I have to understand which variables or properties are not explicitely being assigned by the user.

Update: I think those values are being set in another file; the problem is that I have only the jar file and I'm not able to see the source code of it. As a developer, trying to evaluate a product like jira without being able to see its code, is really a tough task.

In the afternoon I switched to a simplier task. I'm writing a paper about available Jira plugins for agile developers.

10/9/2008

2008-09-10T10:46:00.012+02:00

Task: Try to write a more complicated Jira plugin.

Again, the Jira developer resources SUCK. They provide a developer toolkit without any readme file. They provide plugin examples that don't build. They don't provide any step-by-step tutorial or explanation about how to do things. It's like trying to study marine biology, fishing on a lonely ship in the middle of the ocean.

The provided helper command "mvn eclipse:eclipse"should generate an eclipse project file for a new plugin. Ok, every time I launch it, it downloads more than an hundred MBs from the web. No matter if the files are already in the local repository: it *has* to download them again. I'm getting sick wasting my time waiting for a damn helper tool to finish.

Update: reading in some forum comments (yeah, not in official documentation, in comments!) I found out that jira/atlassian plugins have to be built with the version 1.x of Maven. Amazing. The newest version, 2.0.x, is not working anymore; it's kind of a completely different application. This thing should be written in capital letters with a huge font in EVERY developer page. Instead, it's in a forum comment. Great work, atlassian guys! What a wonderful documentation you have!

Then I tried to build a plugin, this time successfully. However, I can't understand the parameter-passing format of the velocity templates. There's a variable being printed correctly without being assigned anywhere; and assigning manually a value to a variable is not working. Still searching for a solution on this. This should be a i18n (internationalization) variable, and I can't find any documentation about this.

Still stuck on the i18n issue. Posted a question about this on the atlassian jira forum:

http://forums.atlassian.com/thread.jspa?threadID=28566&tstart=0

9/9/2008

2008-09-09T11:05:00.002+02:00

Writing a sample Jira plugin:

Found a great tutorial for a "hello, world" servlet plugin: Sample hello world plugin tutorial
However be careful if you copy'n'paste from this tutorial, cause the syntax is a little messed up from the blog platform.

Maven sucks! At compile time it doesn't check if the xml files provided are valid. It says everything is ok, then when you start the tomcat server you have to discover by yourself what is stopping it from loading, searching in the start log. What a PITA!

After 4 pomodori I finally managed to get a working plugin. This whole thing is a mess. Just to get an "Hello, world!" string on my Jira front page I had to write around 70 lines of code in many different files. Ok, it's only my second day of work on this subject; but the lack of informations (the *simple* ones) is getting me mad.

Here is the link to the jar file I generated. Install it as usual in your jira installation to try it.

8/9/2008

2008-09-09T10:37:00.000+02:00

- Jira installation on local machine and exploration of the application:

- Jira and XP: plugins, graphs. Research for informations about Jira usage in XP environments.

- Example jira plugin. Installed plugin developer toolkit, tried to understand the basic structure of a plugin. Tried to write a sample plugin, without success.

Note: simply installing the developer resources took 2 pomodori: the download process of the needed files was very long.