Daniel Ruoso

Últimos Posts

Categorias

Arquivo

Nuvem de Tags

13.02.2010  Google AI Challenge in Perl

I'm usually not thrilled by competitions, but I must admit this one has taken me. I refer to the Google AI Challenge promoted by the University of Waterloo Computer Science Club. This is a simple bot competition where you play in a one-to-one tron game with another bot from the competition in one of a set of maps.

The deadline for bot submission is 26th february, but don't wait untill there. The example bots are nothing close to the competition bots... You can follow my status here.

Postado por autor: ruoso em Perl.   Tags  competitionPerlaiperl5.

07.02.2010  Writing games in Perl - Part 2 - Controlling the Ball

Following the first post on the subject of writing games in Perl, where we created a bouncing ball (I know, it is a rectangle, but I trust your imagination), this post is going to add something very important when dealing with games: input.

Silveira Neto suggested that I should include more specific instructions on how to start the game (and maybe a video), so I recalled that I didn't mention that all the sources for this posts (including the text) is currently hosted at a github repository (if you plan to contribute, please just ask me for commit permissions instead of forking the repo).

So if you want to run the codes posted here, you first need to:

$ git clone http://github.com/ruoso/games-perl.git

You can check for updates by calling

$ git pull origin master

from inside the games-perl directory. Each directory inside games-perl starts with the number of the post. The first post is inside the 1-bouncing-ball directory and the second is in 2-controlling. To run the the first code just get inside the first directory and call:

$ perl ball.pl

The second example code is based on the first, so the script name is the same, so just get into the other directory and run the same line. If you get an error like:

Can't locate SDL/Video.pm in @INC (@INC contains: /etc/perl
/usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5
/usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10
/usr/local/lib/site_perl .) at ball.pl line 8.

It means you probably don't have the newest SDL, take a look at the first post to see how to get the newest redesigned SDL.

Controlling the Ball

Enough for the introduction, let's get to the actual code. The first thing we need is understanding SDL Events. If you ever programmed GUI applications or even if you wrote some javascript you are aware of how an event framework looks like. SDL is no exception, you need to wait (or poll) for the events, and each event will contain the information you need to figure out what happened.

In our case, we want to apply additional acceleration to the ball whenever the arrow keys are pressed. But if we have an event-based system, the way to figure out which of those four keys is currently pressed is keeping a state mask and update it when you receive keydown and keyup events.

So what we're going to do is to manipulate the acc_h and acc_v ball attributes depending on the keydown and keyup events. It might look complicated, but the only change we need is (this is inside ball.pl main loop):

  while (SDL::Events::poll_event($event)) {
    if ($event->type == SDL_QUIT) {
      exit;

    } elsif ($type == SDL_KEYDOWN &&
             $sevent->key_sym() == SDLK_LEFT) {
      $ball->acc_h(-1);

    } elsif ($type == SDL_KEYUP &&
             $sevent->key_sym() == SDLK_LEFT) {
      $ball->acc_h(0);

    } elsif ($type == SDL_KEYDOWN &&
             $sevent->key_sym() == SDLK_RIGHT) {
      $ball->acc_h(1);

    } elsif ($type == SDL_KEYUP &&
             $sevent->key_sym() == SDLK_RIGHT) {
      $ball->acc_h(0);

    } elsif ($type == SDL_KEYDOWN &&
             $sevent->key_sym() == SDLK_UP) {
      $ball->acc_v(1);

    } elsif ($type == SDL_KEYUP &&
             $sevent->key_sym() == SDLK_UP) {
      $ball->acc_v(0);

    } elsif ($type == SDL_KEYDOWN &&
             $sevent->key_sym() == SDLK_DOWN) {
      $ball->acc_v(-1);

    } elsif ($type == SDL_KEYUP &&
             $sevent->key_sym() == SDLK_DOWN) {
      $ball->acc_v(0);

    }
  }

So, this is it. Follows a small video of the game.

Postado por autor: ruoso em Perl.   Tags  sdlPerlsnippetperl5.

01.02.2010  Writing games in Perl - part 1 - Bouncing Ball

I've been saying I will write a lengthy description of my experiments with SDL in Perl for a while, today kthakore uploaded a SDL experimental version with all the code involved in this experiments. So I decided to start, but I decided to make it a serial post. I'm still unsure about how the format will look like for the rest of the posts, but I certainly would love any comment with ideas of where I should go next.

So at the first post, I'll try to make a very simple application: "bouncing ball". It is simply a ball bouncing in the screen, where the left and right cursors will accelerate them to each side.

Starting the environment

Well, at first you need version 2.3_5 or greater of the SDL module available here. Building the SDL perl binding will require the headers for most of the sdl related libraries. If you're in a Debian machine (or most debian-based distros) you can simply install the regular version of the SDL perl bindings. You can also do:

# apt-get build-dep libsdl-perl

This line requires you to have a deb-src line in your /etc/apt/sources.list. Mine looks like:

deb-src http://ftp.br.debian.org/debian lenny main

In the case you change your sources.list file, remember you need to call apt-get update for it to get the new info.

After that you can simply download SDL 2.3_5 from CPAN and follow the README instructions, which basically means (note that this version is experimental so you might notice some failing tests):

# perl Build.PL
# ./Build
# ./Build test
# ./Build install

You might setup a local::lib if you still want to run other applications that require the older and stable version of SDL (yes, there are a lot of API changes). I'm not a Windows user, but it has been reported that it works like charm when using Strawberry Perl.

If everything is fine, you should be able to get "2.35" when you do:

$ perl -MSDL -E 'say $SDL::VERSION'

You might always get into #sdl@irc.perl.org if you run into trouble.

The way I write games

Alright, before getting into the technicalities of how to write a game with SDL in Perl, let's think a bit on the mechanics of how a game works.

The first thing to realize is that a game needs to simulate some universe, and that this universe needs to have some universal rules like the physics you apply. For instance, it is very important that you decide, from the beggining, if you're going to have gravity in your game.

The second thing to realize is that, even if we have the impression that the time is a continuum, time is actually a sequence of events, like the shutter of a camera in burst mode (okay, for the not interested in photography, think in stop motion). That basically means: in frame A the ball was at position 10,10 and in frame B it was at 20,30. There's no in-between, you don't have to worry about it. You might be wondering if a collision might be lost in that move, but the point is, if the machine can't evaluate enough frames per second as to avoid that, it probably wouldn't be able to evaluate a more ellaborated calculation of the tragetories to see if they would have collided.

This provides an important simplification on how to think your game model. In this first example I'm going to overlook the modelling for interaction with different objects, since we have just a bouncing ball.

One last thing before we go on. You might be tempted to use pixels as your measuring unit, there's one important aspect to keep in mind. A Pixel is an integer value, which means that you'll need to do roundings for each frame. And if you need to store it as a integer value, you're going to accumulate imprecision, which might lead to weird effects, specially when the fps rate changes dramatically. My suggestion is to stick with the good old international measuring system and just use meters, a simple calculation can covert from/to pixels.

package Util;
use strict;
use warnings;
our $DPI = 96; # I think there's a sane way to fetch this value
sub m2px { int(((shift) * ($DPI / 0.0254)) + 0.5) }
sub px2m { (shift) / ($DPI / .0254) }

Modelling the ball

I think there probably isn't a better fit for Object Orientation than games, since you're actually dealing with simulated objects, and the most obvious choice is to use object orientation to work with it. So that's the attributes I can think right now to our object.

package Ball;
use Moose;

use constant g => 1.5;
use Util;
use SDL::Rect;

# how much space does it take
has radius => (is => 'rw', isa => 'Num', default => 0.005);

# Position - vertical and horizontal
has pos_v => (is => 'rw', isa => 'Num', default => 0.1);
has pos_h => (is => 'rw', isa => 'Num', default => 0.04);

# Velocty - vertical and horizontal
has vel_v => (is => 'rw', isa => 'Num', default => 0);
has vel_h => (is => 'rw', isa => 'Num', default => 0);

# Current acceleration - vertical and horizontal
# gravity is added later
has acc_v => (is => 'rw', isa => 'Num', default => 0);
has acc_h => (is => 'rw', isa => 'Num', default => 0);

With our virtual ball defined, we need to implement the simulation of time. And again, we need to think that time is not a continuum, so what we do is providing a "time_lapse" method that will recalculate the attributes of our object according to the ammount of time past.

sub time_lapse {
  my ($self, $old_time, $new_time, $height, $width) = @_;
  my $elapsed = ($new_time - $old_time)/1000; # convert to seconds...

  # now simple mechanics...
  $self->vel_h( $self->vel_h + $self->acc_h * $elapsed );
  # and add gravity for vertical velocity.
  $self->vel_v( $self->vel_v + ($self->acc_v - g) * $elapsed );

  # finally get the new position
  $self->pos_v( $self->pos_v + $self->vel_v * $elapsed );
  $self->pos_h( $self->pos_h + $self->vel_h * $elapsed );

  # this ball is supposed to bounce, so let's check $width and $height
  # if we're out of bounds, we assume a 100% efficient bounce.
  if ($self->pos_v pos_v($self->pos_v * -1);
    $self->vel_v($self->vel_v * -1);
  } elsif ($self->pos_v > $height) {
    $self->pos_v($height - ($self->pos_v - $height));
    $self->vel_v($self->vel_v * -1);
  }

  if ($self->pos_h pos_h($self->pos_h * -1);
    $self->vel_h($self->vel_h * -1);
  } elsif ($self->pos_h > $width) {
    $self->pos_h($width - ($self->pos_h - $width));
    $self->vel_h($self->vel_h * -1);
  }


}

From the simulated universe to pixels

You probably noticed that we haven't talked much about the game development per se, so let's start thinking about it

The first difference from the simulated universe and the actual game is the coordinate system. In our universe the vertical position 0 is near the floor, but in the screen, the vertical position 0 is at the top of the screen.

The other difference, which I already mentioned, is that our measures are in meters, and the screen is measured in pixels. The SDL library provides an important class to interact with this issue, the SDL::Rect.

That way, we're going to have a method to return a rect - but we need to remember to only return a rect that is inside the expected bounds, otherwise SDL will throw an exception.

sub get_rect {
  my ($self, $height, $width) = @_;

  my $inverted_v = $height - $self->pos_v;

  my $x = Util::m2px( $self->pos_h - $self->radius );
  my $y = Util::m2px( $inverted_v - $self->radius );
  my $h = Util::m2px( $self->radius * 2 );
  my $w = Util::m2px( $self->radius * 2 );

  my $screen_w = Util::m2px( $width );
  my $screen_h = Util::m2px( $height );

  if ($x  $screen_w) {
    $w -= ($x + $w) - $screen_w;
  }

  if ($y  $screen_h) {
    $h -= ($y + $h) - $screen_h;
  }

  return SDL::Rect->new( $x, $y, $w, $h );
}

Painting

The last part is actually drawing the ball, which involves painting the ball in the correct place. In our first version, our ball will be a square, since it's the most primitive drawing we have and I don't want to get into that specifics.

my $color;
sub draw {
  my ($self, $surface, $height, $width) = @_;
  unless ($color) {
    $color = SDL::Video::map_RGB
      ( $surface->format(),
        0, 0, 255 ); # blue
  }
  SDL::Video::fill_rect
      ( $surface,
        $self->get_rect($height, $width),
        $color );
}

Yeah, I know, it's an ugly ball, but it bounces... ;)

Putting the pieces together

Now we just need the actual application to use our Ball. Note that the cycle is basically:

  • Listen for events
  • Evaluete the time_lapse
  • Draw the background
  • Draw the ball
  • Update the parts of the screen that where changed
  • Ask for a delay to keep the desired fps rate

Note that I could have asked for it to update the entire screen instead of only the parts that were changed, but that particular operation is actually the most expensive thing you can have in your app, so we try to update as few parts of the screen as possible.

#!/usr/bin/perl

use 5.10.0;
use strict;
use warnings;

use SDL;
use SDL::Video;
use SDL::App;
use SDL::Events;
use SDL::Event;
use SDL::Time;

use lib 'lib';
use Util;
use Ball;

my $ball = Ball->new;
my $width = 0.2;
my $height = 0.15;
my $fps = 60;

my $app = SDL::App->new
  ( -title => 'Bouncing Ball',
    -width => Util::m2px($width),
    -height => Util::m2px($height));

my $black = SDL::Video::map_RGB
  ( $app->format(),
    0, 0, 0 ); # black

my $event = SDL::Event->new();
my $time = SDL::get_ticks;
my $app_rect = SDL::Rect->new(0, 0, $app->w, $app->h);
my $ball_rect = $ball->get_rect($height, $width);

while (1) {
  my $oldtime = $time;
  my $now = SDL::get_ticks;

  while (SDL::Events::poll_event($event)) {
    exit if $event->type == SDL_QUIT;
  }

  my $old_ball_rect = $ball_rect;

  $ball->time_lapse($oldtime, $now, $height, $width);

  $ball_rect = $ball->get_rect($height, $width);

  SDL::Video::fill_rect
      ( $app,
        $app_rect,
        $black );

  $ball->draw($app, $height, $width);

  SDL::Video::update_rects
      ( $app,
        $old_ball_rect, $ball_rect );

  $time = SDL::get_ticks;
  if (($time - $oldtime) < (1000/$fps)) {
    SDL::delay((1000/$fps) - ($time - $oldtime));
  }
}

Postado por autor: ruoso em Perl.   Tags  sdlPerlsnippetperl5.

08.01.2010  My quest to SDL in Perl

I think this thing started when Silveira Neto posted his code snippet on Pygame animating the tile set he has been creating for a long time. I knew for a long time that Perl supported SDL and that Frozen Bubble is written in Perl, but I have never tried it myself.

Working with graphics wasn't new to me, back in the days I played with BASIC, I wrote a blackjack program with animated cards, some time later - already in Linux, I played a bit with vgalib. I've been also thinking for a while on the concept of how to evaluate the progress of time in a game environment.

If you join this with the fact that I now have a 1yo kid, it kinda urged me to writing something in SDL for him. The idea was quite simple, as he doesn't coordinate mouse and keyboard movement yet, the game would simply present random images and sounds as there was any event at all.

So I started by using a simple event loop for the SDL events, and a timer to evaluate the time lapse and draw the game. To my surprise, using the timer caused segfaults, because the timer callback runs in a different thread in SDL, but the binding didn't provide any support for it and all I got was kaboom.

I worked around this issue by using Event, a simple event library I already used some other places, to implement the timer. It worked like charm, as there was events in SDL, it was creating rects of random sizes and colors that would last for a random amount of time, fading to black.

Then I thought: Ok, now let's proceed to audio. To my surprise, the callbacks for procedural audio generation were simply missing - Frozen Bubble only plays pre-produced audio files - and my idea was to produce random audio (not noise, but sounds in random frequencies). This was the push I needed to get to the redesign branch in github and start porting the app to it.

Lots of research later, with the help of kthakore++ and the tips from Nicholas++, we managed to have a working version of SDL based on threads and threads::shared. This was a very great moment. I must confess I always kept some envy from python people because of the - so I thought - better support for threads in Python. At some point in this path I found out that Python doesn't support threads at all, because they have a Global Interpreter Lock, which basically means that POE implements the same kind of scalability than Python (and it also explained why the people at ce.gov.br have so much trouble with zope scalability). So now not only we have a thread-enabled SDL binding, but it is *really* threaded, which means that the audio callback will really run at the same time as the timer callback , which will also run together with the event loop. And now I'm happy to know that Perl does have a better threading support than Python - yeah, I know envy is a bad thing, but... whatever...

After I finished Tecla, the app for my 1yo kid, I decided to play with Game::PerlInvaders, which I saw some time ago and I knew some things I could improve. The result is in the branch refact_ruoso at github.

I should make another post, with the details on how tecla and the rewrite of Game::PerlInvaders work, but that's it for now.

Postado por autor: ruoso em Perl.   Tags  sdlPerlsnippetperl5.

14.12.2009  Analog ASCII Clock

Today I started the day looking at this post that inspired me to write an ascii analog clock in Perl.

Unlike the linked post, I didn't intend to golf, but rather writing a clean and readable code.

Follows the code:

#!/usr/bin/perl

# besides DateTime, the standard pragmas
use 5.010;
use strict;
use warnings;
use utf8;
use DateTime;

# initialize configuration variables
my $width = 32;
my $height = 32;
my $mark_at = 15;
my $length_m = 12;
my $length_h = 8;
my $length_s = 15;
my $pi = 3.14159265;
my $center_x = int(0.5 + ($width / 2));
my $center_y = int(0.5 + ($height / 2));
my $loop = 1;

# clear the screen and position the cursor in the left top corner
print "\x1b[0;0H\x1b[2J";

# loop the rendering of the clock
while ($loop && sleep 1) {

  # create a matrix of the configured height and width with blanks
  # I use two spaces because most console fonts have the height with
  # twice the width
 my $data = [map {[ map { '  ' } 1..$width ]} 1..$height];

  # get the date and time in the local time zone
  my $now = DateTime->now(time_zone => 'local');
  my $hour = $now->hour_12;
  my $min = $now->minute;
  my $sec = $now->second;

  # convert that to degrees for the hands of that clock
  my $degree_m = ($min * 6) + 270;
  my $degree_h = ($hour * 30) + 270;
  my $degree_s = ($sec * 6) + 270;

  # see the place sub down there
  # place the numbers 1 to 12 around the clock
  place($data,($_ * 30) + 270,$mark_at,sprintf("%02d",$_)) for 1..12;

  # now the seconds hand
  place($data,$degree_m, $_ , '<>') for 0..$length_m;
  
  # the hours hand
  place($data,$degree_h, $_ , '::') for 0..$length_h;

  # the seconds hand
  place($data,$degree_s, $_ , '..') for 0..$length_s;

  # mark the center o the clock
  $data->[$center_y][$center_x] = 'OO';

  # position the cursor in the left top corner
  print "\x1b[0;0H\n";
  
  # covert the matrix into a single string and print it
  say join "\n", map { join '', @$_ } @$data;
};

# covert degrees and distance into X and Y coords using basic trigonometry
sub vec2xy {
  my ($deg, $len) = @_;
  return $center_x + int(0.5 + ($len * cos($deg * ($pi/180)))),
    $center_y + int(0.5 + ($len * sin($deg * ($pi/180))));
}

# place a string into some degree and distance
sub place {
  my ($data, $deg, $len, $txt) = @_;
  my ($x,$y) = vec2xy($deg,$len);
  $data->[$y][$x] = $txt;
}

Postado por autor: ruoso em Perl.   Tags  Perlsnippetperl5.

30.08.2009  Missing :ignoreaccent in Perl 5

So I'm working with natural language processing, and one of the steps in that is tokenization. For those not familiar with the term, it means taking an "expression" and splitting it into the several "parts" of it. In SQL, for instance SELECT * FROM foo WHERE bar='baz' is composed by the tokens 'SELECT', '*', 'FROM', 'foo', 'WHERE', 'bar', '=', '"baz"'. This is not much surprising, a very naive look would say that I could split in word boundaries, but that's just not the case, since the quotes can include spaces inside it, and can include escape characters inside that. But it's still a very simple and very well documented syntax, so tokenization is fairly simple. In fact, you can look at SQL::Tokenizer from the fellow brazillian monk izut.

But unlike SQL, natural languages are everything but simple and well documented, and there are no conformance requirements, because the text is intended for another human being to parse it, not a machine. During this project I realized I was a bit behind in the literature, since I was still using Chomsky as my reference, and the linguistics field already flew apart from that, now accepting that grammars "statistically emerge" from the language use and that a "top-down" approach, which is to define a general grammar model and use it to classify the text is not that much usefull in the long term, since the language use evolve too fast and its variability depending on the media in use, as well as the social environment create conflicting definitions of the grammar. Any resemblance with Perl 6 hability to have custom grammars changing the way the code is parsed is not a mere coincidence.

So, the problem with tokenizing natural language texts, is, at first, the locale problem. In german "ß" would be normalized to "ss", while in greek, it would be "normalized" to "b" and in other languages it should probably be ignored as noise (yes, people often use foreign characters as bullet markers or other decorators). Of course I could do a two-phase tokenization, first doing a naive tokenization to discover the language, then doing a second presuming the locale, but I actually decided to do something alledgedly more clever, which is to ignore the accents when trying to remove the non-important characters, so I can do a simple \W match to remove the non-alpha characters.

The thing I miss in Perl 6 is that you could just use the :ignoreaccent modifier so the match would already match against the base character. What is a simple regex modifier in Perl 6, in Perl 5 needs to be done as:

use strict;
use warnings;
use Text::Unaccent;
use utf8;
use Encode;
use 5.10.0;

my $str = 'têmó�›... åçèñŧos!!!';
my $unac = unac_string('utf8',encode('utf8',$str));
my $d = $unac;
(my $words = $unac) =~ s/(\W)/substr($d,$-[0],1,' ')/ge;

say $str;
say $unac;
say $words;
say $d;

And with that code, I have a much easier task when tokenizing natural languages...

Postado por autor: ruoso em Perl.   Tags  Perlsnippetperl6perl5.

18.08.2009  Transactions and Authorization made simple

So I really like to follow DRY: Don't Repeat Yourself. In the development of Epitafio (A cemetery management system I mentioned earlier), I was workin on my model classes - note that this is not a DBIC model, but a regular model that do access a DBIC schema - and I realized that for every single method of the models I would need to do two things:

  • Enclose code in a transaction, much like:
    $schema->txn_do(sub { ... })
  • Authorize the user against a specific role:
    die 'Access denied!' unless $user->in_role('foo')

So I started wondering at #catalyst if there would be a pretty way of doing it. I was already using Catalyst::Component::InstancePerContext, but mst quickly guided me to avoid saving the context itself in the object, but rather getting the values I need from there. Since my app models will basically follow this same principle I did a model superclass with:

package Epitafio::Model;
use Moose;
with 'Catalyst::Component::InstancePerContext';
has 'user' => (is => 'rw');
has 'dbic' => (is => 'rw');

sub build_per_context_instance {
  my ($self, $c) = @_;
  $self->new(user => $c->user->obj,
             dbic => $c->model('DB')->schema->restrict_with_object($c->user->obj));
}
1;

Note that I'm still using the C::M::DBIC::Schema as usual, but I'm additionally making a local dbic schema that is restricted according with the logged user. Check DBIx::Class::Schema::RestrictWithObject for details on how that works, and mst++ for the tip.

Ok, now my model classes can know which user is logged in (in a Cat-independent way) as well as have access to the main DBIC::Schema used in the application. Now we just need to DRO - Don't Repeat Ourselves.

Following, again, mst++ tip, I decided against doing a more fancy solution and gone to a plain and simple:

txn_method 'foo' => authorize 'rolename' => sub {
   ...
}

For those who didn't get how that is parsed, this could be rewritten as:

txn_method('foo',authorize('rolename',sub { }))

This works as:

  • authorize receives a role name and a code ref and returns a code ref that does the user role checking before invoking the actual code.
  • txn_method receives the method name and a code ref and installs a new coderef that encloses the given coderef into a transcation in the package namespace as if it were a regular sub definition.

That means you can have a txn_method without authorization, but you would require

our &foo = authorize 'rolename' => sub { ... }

to get authorization without transaction. But as in my application I'll probably have both most of the time, I thought it should suffice the way it is.

But for the txn_method..authorize thing to parse, both subs need to be in the package namespace at BEGIN time, so to solve that, without having to re-type it every time, I wrote a simple Epitafio::ModelUtil module that exports this helpers.

package Epitafio::ModelUtil;
use strict;
use warnings;
use base 'Exporter';

our @EXPORT = qw(txn_method authorized);

sub txn_method {
  my ($name, $code) = @_;
  my $method_name = caller().'::'.$name;
  no strict 'refs';
  *{$method_name} = sub {
    $_[0]->dbic->txn_do($code, @_)
  };
}

sub authorized {
  my ($role, $code) = @_;
  return sub {
    if ($_[0]->user->in_role($role)) {
      $code->(@_);
    } else {
      die 'Access Denied!';
    }
  }
}

1;

And now the code of the model looks just pretty and non-repetitive ;). See the sources for the full version.

Postado por autor: ruoso em Perl.   Tags  epitafioPerlperl5catalyst.

12.08.2009  SMOPP5 first steps

After a long time imagining when this day would come, today Paweł Murias has created a github fork of the perl interpreter so we can start working on the integration of SMOP and perl5.

Some of you might have heard me saying that the major reason for SMOP to exist today is the prospective integration with the perl5 interpreter so we can use Perl 6 at the same time as still being able to use all of CPAN, including the things that depend on XS, like the fantastic Gtk2-Perl suite.

In fact, I've been blocking pmurias on some things like replacing the refcounting by a trace gc in SMOP exactly because that would make SMOP incompatible with perl5, and I really want them to cooperate.

This integration should happen at the deepest level of perlguts, where the perl5 interpreter should play the role of the SMOP interpreter and every SV* is also a SMOP__Object*.

Paweł has added smop/base.h to the p5 repo and I started adding the SMOP__ResponderInterface* member to some p5 values (right now _SV_HEAD, which defines the first members of every SV value, and the PerlInterpreter). This is the first step that will allow SMOP to use P5 objects without the need for a proxy value.

After talking with nothingmuch on #p5p, I decided to note here the first set of goals of the SMOPP5 integration:

  • Making every perl value a SMOP__Object*
  • Implemeting Responder Interfaces for each of this values
  • Implementing the SMOP interpreter and continuation class APIs in the perl5 interpreter (using Coro::State for now)
  • Have SMOP objects visible in perl5 using proxy objects as already happens today

This set gives use the SMOP->P5 integration, after that we're going to need the P5->SMOP integration, which should involve hacking in every p5 macro in the core, which is a *lot* of hacking, so I'll not include it as our goals for now, for sanity sake!

Postado por autor: ruoso em Perl.   Tags  Perlsmopp5smopperl6perl5.

11.08.2009  Far More Than You Ever Wanted To Know About Typeglobs, Closures and Namespaces

This are the slides of a presentation I gave at a tech meeting in Lisbon about 2 years ago. The slides text is in portuguese, but I'm pretty sure they are understandable for non-portuguese speakers too.

Far More Than You Ever Wanted To Know About Typeglobs, Closures and Namespaces

Postado por autor: ruoso em Perl.   Tags  Perlfmtyewtkslidesperl5.

31.07.2009  Too much Perl 6

So, yesterday I was giving a quick perl workshop using Catalyst. The idea was to write a blog in 3 hours. At some point I wrote the following code:

sort { (stat $_)[10] } glob 'posts/*';

And it didn't work, because Perl 5 doesn't DWIM when I have a sort routine that takes only one parameter, while Perl 6 would realize that and use that value for a later sorting. Basically, Perl 6 implemented the schwartzian transform in its core.

Postado por autor: ruoso em Perl.   Tags  Perlperl6perl5.

30.07.2009  Epitáfio

The Perl monks from Brazil accepted a challenge with an important social relevance. We are working, on our spare time, in the development of a system to manage the public cemeteries. Few people know, but the public cemeteries take a fundamental role regarding the respect for human rights, as well as for public health, giving the population that doesn't have the resources to pay for a tombstone in a private cemetery a memorial service and a decent burial.

At this moment we already stablished the features for the first release, and the deadline expectation is that we have the system working in the cemetery for the "dia de Finados" (day in memory of the people who past away). Yesterday we finished the first proposal for the data model (the software will be developed in Portuguese, but I guess you can figure out the meaning of the words).

The discussions regarding the system happen in the #brasil-pm channel at irc.perl.org, we have a space in the perl.org.br wiki to document the development process and a github space to host the source code.

The system will have a Web interface and is going to use PostgreSQL, specially because of the timestamp-related features and also for possibly using PostGIS, allowing to store the spatial information about the tombstones and the cemetery map.

As the development follows, I'll post the updates here.

Postado por autor: ruoso em Perl.   Tags  epitafioPerldireitos humanosperl5.

20.07.2009  Implementing SOAP in Perl today

I started looking at SOAP around 2001, and at that time SOAP::Lite was the only viable option to both consume and produce SOAP services in Perl. Everybody knew SOAP::Lite was a mess, but it was pretty much what we got and we were stuck with it.

In 2001 SOAP::Lite wasn't much a problem and that's because it implements the RPC/Encoded format for SOAP messages, and by that time this was pretty much the default. But the usage of SOAP changed, a lot, and the module didn't catch up with that changes.

First of all, we need to understand that SOAP is much more than a way to do RPC, if you need plain RPC without much management, then XML::RPC or JSON::RPC is just fine and will work without any concerns. But if, on the other hand, you need a strict management of the data being transfered as well as a proper documenting of which services use which data types, then SOAP is probably the thing you need.

But for using SOAP, there are two aspects you need to undestand:

Style

The style describes the general semantics of the service. There are two different styles in SOAP:

  • Document: This style is used for services that represent the submission of some arbitrary data that expects a result or not. One example of a service where Document style makes sense is to submit a purchase in a B2B system.
  • RPC: This style is used where you're acessing a procedural or oriented object API of some sort, its semantics define that one specific resource support several operations.

The biggest practical difference is that in the Document Style, you're going to submit one or more XML documents as the message, and this document is the resource to be used by that specific operation. In the RPC style, the first and only child of the body element describes which is the operation that this message is trying to execute, and the parameters for that operation are the direct child of that main element.

Body Use

Indpendent of the style of the service, the other aspect that governs how SOAP works is the "use" of the body. There are two types of "use"

  • Encoded: This referes to SOAP-Encoding, which is a specific format for data serialization described by the SOAP protocol itself. In the early usage of SOAP, this was the main way of exchanging data.
  • Literal: This defines that the content of the message is actually encoded acording to some arbitrary XML Schema, this has the advantage of being able to represent any data that XML can, and also makes the serialization format more decoupled from the language providing the service.

Style/Use

Considering all that, there are four ways of using SOAP:

  • RPC/Encoded: This is the only format supported by SOAP::Lite, and hardly used in new services.
  • RPC/Literal: This allows you to use arbitrary XML data as the parameters and response for an API that you want to expose.
  • Document/Encoded: While this mix is theoretically possible, I have never heard about any use of it.
  • Document/Literal: This is a type of service where you have only one operation per endpoint, and usually don't map well to RPC semantics, since it would require several endpoints to implement the whole API.

WARNING: Microsoft created a pseudo-standard called Document/Literal-Wrapped which is a mix of both RPC/Literal and Document/Literal. The service is described with Document/Literal, but the XML Schemas of the service are made considerably more complex to specify the wrapper element that would be otherwise natural in the RPC Style. It also requires the use of the SOAP-Action header to define which operation is being called. The SOAP-Action header is HTTP specific and was supposed to be used for routing purposes, not to define which is the operation being invoked. Please use RPC/Literal when you need RPC semantics with Literal body use, although I have implemented a compatibility mode for this aberration, it should not be promoted in any way.

Finally, let's implement some SOAP.

The first thing you need to do when implementing a SOAP service is deciding wether your service has RPC or Document semantics. In our example, I'm going to implement a Document-style service "converter", which receives a document describing the amount, the origin unit and the target unit, returning a document with the target amount set.

An example of a RPC style service would be an API that would include things like: get_available_source_units(), get_available_target_units() and other correlated operations. In our case we are going to assume that the document itself provides all the information we need and that the associated metadata (the XML Schema) will provide us sane data.

The second think you need to do is describing your data in the XML Schema format, so for our "unit converter" service, we're going to have a UnitConversion element. If you're used to XML, you know that you need a namespace URI for your data and your services, I'm going to use http://daniel.ruoso.com/categoria/perl/soap-today as the namespace for both the data and the service.

<xsd:schema elementFormDefault="qualified"
 xsd:targetNamespace="http://daniel.ruoso.com/categoria/perl/soap-today"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:unit="http://daniel.ruoso.com/categoria/perl/soap-today">
 <xsd:element name="UnitConversion">
  <xsd:complexType>
   <xsd:sequence>
    <xsd:element name="input">
     <xsd:complexType>
      <xsd:sequence>
       <xsd:element name="amount" type="xsd:double" />
       <xsd:element name="unit" type="xsd:string" />
      </xsd:sequence>
     </xsd:complexType>
    </xsd:element>
    <xsd:element name="output">
     <xsd:complexType>
      <xsd:sequence>
       <xsd:element name="amount" type="xsd:double" />
       <xsd:element name="unit" type="xsd:string" />
      </xsd:sequence>
     </xsd:complexType>
    </xsd:element>
   </xsd:sequence>
  </xsd:complexType>
 </xsd:element>
</xsd:schema>

The above XML Schema will support data that looks like:

 <UnitConversion>
  <input>
   <amount>10</amount>
   <unit>kg</unit>
  </input>
  <output>
   <unit>pound</unit>
  </output>
 </UnitConversion>

Now that you specified how your data looks like, you need to specify how your service looks like that is done using the Web Service Description Language -- WSDL. Basically, you define:

  • Messages: The messages describes a single message exchange, independent of the Message Exchange Pattern. The WS-I says that you should have "Request" and "Response" as part of the message names, but I disagree, since a message might be used for both request and response.
  • Port Type: The Port Type describes the interface of the service, independent of the transport, basically grouping the messages into a specific "Message Exchange Pattern" -- usually this is request/response, but you can have both request-only and reponse-only.
  • Binding: The binding associates a given port type with some transport mechanism and describes the attributes of how that message will be transported. It's in the binding that you'll describe you want to use SOAP as well as the style/use.
  • Service: The service groups a set of bindings defining where are the endpoints for the client to access the services.

I'm not going to explain each detail of the WSDL here, but I think it's pretty straight forward. WSDLs are complex when generated by tools like .Net, but it doesn't need to be. This WSDL assumes that the previous XML Schema is saved as unit-conversion.xsd.

<wsdl:definitions
  xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
  xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
  xmlns:unit="http://daniel.ruoso.com/categoria/perl/soap-today"
  targetNamespace="http://daniel.ruoso.com/categoria/perl/soap-today">
 <import namespace="http://daniel.ruoso.com/categoria/perl/soap-today"
         uri="unit-conversion.xsd" />
 <wsdl:message name="ConvertUnit">
  <part name="UnitConversion" element="unit:UnitConversion" />
 </wsdl:message>
 <wsdl:portType name="ConvertUnit">
  <wsdl:operation name="convert_unit>
   <wsdl:input message="unit:ConvertUnit" />
   <wsdl:output message="unit:ConvertUnit" />
  </wsdl:operation>
 </wsdl:portType>
 <wsdl:binding name="ConvertUnitSOAPHTTP" type="unit:ConvertUnit">
  <soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
  <wsdl:operation name="convert_unit>
   <wsdl:input>
    <soap:body use="literal">
   </wsd:input>
   <wsdl:output>
    <soap:body use="literal">
   </wsdl:output>
  </wsdl:operation>
 </wsdl:binding>
 <wsdl:service name="ConvertUnitService">
  <wsdl:port name="ConvertUnit" binding="unit:ConvertUnitSOAPHTTP">
   <soap:address location="http://localhost/myservice" />
  </wsdl:port>
 </wsdl:service>
</wsdl:definitions>

Ok, now let's really implement it

Now that we're gone through all the overhead of SOAP (and I say it again: if you think this overhead is overkill, go use XML::RPC or JSON::RPC instead, but if you need the services to be strictly documented, SOAP is what you need) we can implement the service itself. And for that we're going to use Catalyst::Controller::SOAP.

I'm not going to explain how to create a Catalyst application, there are lots of tutorials on how to do it, and if you're still reading this, you're probably aware of Catalyst already. So, after you create your Catalyst application, you need a controller that will implement the service, all you need to do is subclass Catalyst::Controller::SOAP and implements the service itself.

package MyApp::Controller::UnitConverter;
use strict;
use warnings;
use base qw(Catalyst::Controller::SOAP);

__PACKAGE__->config->{wsdl} =
  {wsdl => '/usr/share/unit-converter/schemas/UnitConverter.wsdl',
   schema => '/usr/share/unit-converter/schemas/unit-conversion.xsd'};

sub convert_unit :WSDLPort('ConvertUnit') {
    my ($self, $c, $unit_conversion) = @_;
    my $data = $unit_conversion->{UnitConversion};
    if ($data->{input}{unit} eq 'kg' &&
       $data->{output}{unit} eq 'pounds') {
      $data->{output}{amount} =
        2.20462262 * $data->{input}{amount}
      $c->stash->{soap}->compile_return($unit_conversion);
    } else {
      $c->stash->{soap}->fault({ code => 'SOAP-ENV:Client', reason => 'unsupported' });
    }
}

1;

And voilá! You have a SOAP service running. Please refer to the Catalyst::Controller::SOAP docs for more information on the details or just ask me, either in the comments or at #catalyst@irc.perl.org

Postado por autor: ruoso em Perl.   Tags  Perlsnippetsoapperl5web services.

05.07.2009  How do we get out of this mess?

The Perl vs DarkPAN issue is evolving a lot these days, I'd like to explore a viable option so we get out of this mess.

Let me just review which are the options that are clearly not acceptable by one side or the other:

  • Have the defaults for Perl 5 changed, in a backward-incompatible way, favoring Modern Perl
  • Enforce backward-compatibility in order to preserve the huge amount of software running in Perl around the world.

It seems pretty clear that going to each extreme will not build any consensus, so, I was thinking about the way "perl -E" works on 5.10, and I was wondering if we could do one of:

  • Have the defaults changed, but with a switch that would enable "backward-compatible mode"
  • Have the defaults kept, but with a switch that would enable "modern-perl mode"

Basically, what I'm proposing is that we assume that there are two Perls in perl, and by a switch you can select the one you're going to work with. Even if the "modern-perl mode" is assumed to be default, simply adding a switch to the invocation of the interpreter in the old systems wouldn't be much hard.

Postado por autor: ruoso em Perl.   Tags  Perldarkpanbackward compatibilityperl5.

01.07.2009  Perl 5 backward compatibility is not only for DarkPAN

NPEREZ has written that we shouldn't care about the DarkPAN, Both AdamK and SF had argued about it, but I'd like to take it to another perspective.

One of the key strengths of both Perl and perl, has been the way old code is still supported in the language. That is a decisive aspect as when to choose a language to invest a lot of time, and by that, money. I'll take an example that is not part of the DarkPAN and that has been, just last week, released as free software.

About a year ago, I started working in a software, and of course, I chose to write it in Perl. More than that, I have also made some interesting choices, that including the use of SOAP over XMPP to design the choreographies of the application, as well as the use of Catalyst to implement that. That led me to write some new CPAN modules. Basically I was using XML::Compile, XML::Compile::Schema and XML::Compile::SOAP as the foundation to the entire application.

As the software evolved, a new major release of the XML::Compile framework was released, and it mostly changed the API and a lot of default behaviors. This change costed me about 3 months in investigation and patches, both to my code, to XML::Compile's code and to some other modules just to have the code working again.

Imagine what happens if the defaults for the core changes? That would represent an incredibly higher number of issues to be dealt with, and we don't actually have a choice of "sticking with the old perl", because the developers usually are not the sysadmins, and having to convince a sysadmin that you need to install an old, unsupported version of a CPAN module is already bad, imagine having to convince a sysadmin to install an old, unsupported version of perl? and by that it would mean to keep an entire module tree specific for that version?

That would certainly be a reason for a manager to decide to switch over from Perl to something else, and a very good one. In fact, one of the key strengths of Perl is precisely the long backward compatibilty. And that is not only important only to the DarkPAN, but for every single piece of legacy code in the free software ecossystem as well.

Postado por autor: ruoso em Perl.   Tags  Perldarkpanbackward compatibilityperl5.

17.06.2009  Serial communication in Perl

João Gabriel asked in cascavel-pm:

Is there a possibility of using a Perl script (in a Linux environment) to send and receive data from a parallel port (/dev/ttyS0)? Did someone do that with Perl already? Should I adventure myself in Perl or should I jump to C?

Well, you just listed the most evident case of when programming for Linux is easier that when programming to Windows. In summary, the serial port is a file you read from and write to. It's worth to mention that /dev/ttyS0 is a serial port, not parallel. The parallel port is usually /dev/lp. They are fundamentally different, as one implements the RS232 protocol to encode a given stream of information, while the parallel port is just a set of pins you can manipulate. Well, that being said, let's go to the interesting part... First of all, you need to open the port, and to do that:
  use IO::Handle; # so you can use OO API with filehandles
  open my $fh, '+<', '/dev/ttyS0' or die $!;
The '+<' open mode means that you're openning it for both reading and writing (perldoc -f open is your friend). After that, you probably want non-blocking IO, except if your protocol os considerably predictable (it never is). To do that:
  $fh->blocking(0);
It might be necessary to configure the serial port, since the device might not be using the same default configuration as you are. In oreder to do that you need access to the POSIX API, specifically termios.h. So, the first thing you need is:
  use POSIX qw(:termios_h);
Then, to configure the port you can do something like:
    my $term = POSIX::Termios->new;
    $term->getattr(fileno($fh)) or die $!;
    $term->setiflag( $term->getiflag &
      ( &POSIX::IGNBRK | &POSIX::IGNPAR &
        ~&POSIX::INPCK & ~ &POSIX::IXON &
        ~ &POSIX::IXOFF));
    $term->setlflag( $term->getlflag &
     ~( &POSIX::ICANON | &POSIX::ECHO |
        &POSIX::ECHONL | &POSIX::ISIG |
        &POSIX::IEXTEN ));
    $term->setcflag( $term->getcflag &
      ( &POSIX::CSIZE | &POSIX::CS8 & ~&POSIX::PARENB));
    $term->setospeed(&POSIX::B1200);
    $term->setispeed(&POSIX::B1200);
    $term->setattr(fileno($fh), &POSIX::TCSANOW) or die $!;
perldoc POSIX and man termios (install manpages-dev) help you understand what it does. Now you're ready to read and write from/to the serial port, but unless you have a very controlled environment, you don't want to do that in a synchronous way. And for that I recommend using the EV module, which will give you all the tools you need to do the async communication.
Postado por autor: ruoso em Perl.   Tags  Perlserial porttermiosevasyncperl5io.