Matthias Günther

My Talk at Ruby Midwest 2013

2013-05-02T00:00:00+02:00

Here is a picture that explains everything I experienced at this conference:

I received love, motivation, fun, feeling that I’m doing right, passion, energy, and new ideas. But most importantly I made new friends.

The topic of my talk: “We all love Open Source work and Pomodoro will help you to hack on your beloved babies in a time-boxed way to get more done in less time.”

Slides

You can find the slides on speakerdeck.

Video

You can find this video on confreaks.

Impressions

More pictures can be found under flickr Ruby Midwest 2013

Notes

Software: impress.js
Typface:
- FFFTusj
- Lobster Two
Source: Bitbucket

Speaking at Ruby Midwest 2013

2013-02-12T00:00:00+01:00

I received notification that my talk proposal for Ruby Midwest 2013 has been accepted. My talk “More time for Open Source work with the help of the Pomodoro Technique” will show you how you can do more Open Source work besides your day job and other liabilities. I’m very excited, as this will be my first time giving a talk at a conference.

Ruby Midwest 2013 will be held in Kansas City, 5th and 6th of April. Michael Feather and James Edward Grey II are giving keynotes and I will receive the opportunity to meet them. If you attent the conference as well, let us get in touch.

Getting started with RSpec Puppet

2012-09-27T00:00:00+02:00

You never improve if you can't change at all.
Tom DeMarco

Writing your service configuration with Puppet can be easy. But when it comes to debugging, it can be very difficult to edit things and to find certain errors. This article presents your the basic of setting up your environment for testing Puppet modules in a BDD way.

The tools

The rspec-puppet gem is the Gem setup to get started. It install the rspec-puppet-init command which automatically sets up the basic settings for testing. As a text tool we want to use puppet-lint: This tool checks your Puppet manifests against the Puppet Labs style guide and alert you to any discrepancies => you have now your constant feedback loop when writing code.

Run the following commands to install the gem:

$ gem install rspec-puppet
$ gem install puppet-lint

You can use puppet-lint in your terminal to check a puppet manifest.

$ puppet-lint <path-to-your-manifest>

Let’s assume, we have the following puppet file:

# manifests/init.pp

class git::init {
 include git:: package
}

If we run puppet-lint on it:

$ puppet-lint manifests/init.pp
ERROR: two-space soft tabs not used on line 10
WARNING: unquoted resource title on line 10

Setting up the environment

The next step is to clone the puppet boilerplate repository. It’s perfekt for creating a initial skeletton for a new module . After we get the code, we run a script which guides you through the process of creating a new module:

$ git clone git://github.com/bitkollektiv/puppet-boilerplate-modules.git
$ ./puppet-boilerplate-modules/newmodule.sh

Answer the questions in this dialog, that means select the module name, the template for it (0: application-001 is perfect for the beginning), the location, and the author. When your are done with this, go into the directory of your new module and perform the following cleanup commands:

$ cd <your-module-path>
$ rm COPYING CREDITS Modulefile NOTICE README
$ rm -rf files/ templates/

The cleanup is necessary to get you focused on the basics of testing (of course documentation, README, and so on are important but not when you are going to learn something new). Now, your file structure should look like the following.

`-- manifests
    |-- init.pp
    |-- package.pp
    `-- params.pp

This structure follows the package, config, and service (okay, we have params.pp instead of service.pp but this not bad because the module we create in this example isn’t a service) pattern as mentioned by R.I.Pienaar blog post.

The last step is to run rspec-puppet-init in the directory of your module and it will create all the files for testing.

$ cd <your-module-path>
$ rspec-puppet-init
 + spec/
 + spec/classes/
 + spec/defines/
 + spec/functions/
 + spec/hosts/
 + spec/fixtures/
 + spec/fixtures/manifests/
 + spec/fixtures/modules/
 + spec/fixtures/modules/git/
 + spec/fixtures/manifests/site.pp
 + spec/fixtures/modules/git/manifests
 + spec/spec_helper.rb
 + Rakefile

And the file structure should be the following:

|-- manifests
|   |-- init.pp
|   |-- package.pp
|   `-- params.pp
|-- Rakefile
`-- spec
    |-- classes
    |-- defines
    |-- fixtures
    |   |-- manifests
    |   |   `-- site.pp
    |   `-- modules
    |       `-- git
    |           `-- manifests -> ../../../../manifests
    |-- functions
    |-- hosts
    `-- spec_helper.rb

The symlinks in the spec/ directory are linking the manifests folder into your spec folder, so that they are in the runpath of your specs when you run rspec.

Testing, testing, and testing

First, we want to test that the class git::package is created in manifests/init.pp manifests. All we need to do is to create a spec named init_spec.rb in the spec/classes directory. To make it testable, we need to define a scope for our init.pp manifest

# manifests/init.pp

class git::init {
  class { 'git::package': }
}

Let’s do the test for it:

# spec/classes/init_spec.pp

require 'spec_helper'

describe "git::init" do
  it { should create_class('git::packagee')}
end

Now it’s time to run our test: (Remember to run rake spec always from the root directory of your module)

$ cd <your-module-path>
$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/init_spec.rb

git::init
  should contain Class[git::packagee] (FAILED - 1)

Failures:

  1) git::init
     Failure/Error: it { should create_class('git::packagee')}
       expected that the catalogue would contain Class[git::packagee]
     # ./spec/classes/init_spec.rb:4:in `block (2 levels) in <top (required)>'

Finished in 0.05637 seconds
1 example, 1 failure

Failed examples:

rspec ./spec/classes/init_spec.rb:4 # git::init

Prevention of bad code with tests.

Duh, it’s red, what should we do? The catalogue does not contain a class git::packagee. Gosh, it’s just a typo in our init_spec.rb file. Let’s fix this:

# spec/classes/init_spec.pp

require 'spec_helper'

describe "git::init" do
  it { should create_class('git::package')}
end

And run our tests again:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/init_spec.rb

git::init
  should contain Class[git::package]

Finished in 0.03963 seconds
1 example, 0 failures

It’s green and running – perfect.

Testing the creation of a package with a type

Since we are now sure, that the package manifests is integrated, it’s time to write a test, that we have the git-core package in our package manifests. Let’s write spec/classes/install_spec.rb:

# spec/classes/package_spec.pp

require 'spec_helper'

describe 'git::package' do

  context 'install git-core' do
    it { should contain_package('git-core')
    }
  end

end

The contains_<resource> matcher will test if the manifest contains a particular puppet resource.

And run the tests again with rake spec form the root directory of your module:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/package_spec.rb spec/classes/init_spec.rb

git::package
  install git-core
    should contain Package[git-core] (FAILED - 1)

git::init
  should contain Class[git::package]

Failures:

  1) git::package install git-core
     Failure/Error: it { should contain_package('git-core')
       expected that the catalogue would contain Package[git-core]
     # ./spec/classes/package_spec.rb:6:in `block (3 levels) in <top (required)>'

Finished in 0.19843 seconds
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/package_spec.rb:6 # git::package install git-core

Let’s edit manifests/package.pp file:

# manifests/package.pp

class git::package {
  package { 'git-core':}
}

And run the tests again:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/package_spec.rb spec/classes/init_spec.rb

git::package
  install git-core
    should contain Package[git-core]

git::init
  should contain Class[git::package]

Finished in 0.19894 seconds
2 examples, 0 failures

Next we want to add the ensure attribute to the get the latest version of the git-core package. Let’s write a failing test first:

# spec/classes/package_spec.pp

require 'spec_helper'

describe 'git::package' do

  context 'install git-core' do
    it { should contain_package('git-core')
         .with_ensure('latest')
    }
  end

end

The with_* and without_* matcher can test the presence or absence of the parameter of resources. Run our tests, to see they are failing:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/package_spec.rb spec/classes/init_spec.rb

git::package
  install git-core
    should contain Package[git-core] with ensure => "latest" (FAILED - 1)

git::init
  should contain Class[git::package]

Failures:

  1) git::package install git-core
     Failure/Error: .with_ensure('latest')
       expected that the catalogue would contain Package[git-core] with ensure set to `"latest"` but it is set to `nil` in the catalogue
     # ./spec/classes/package_spec.rb:7:in `block (3 levels) in <top (required)>'

Finished in 0.19862 seconds
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/package_spec.rb:6 # git::package install git-core

Time to fix it:

# manifests/package.pp

class git::package {
  package { 'git-core':
    ensure => latest
  }
}

If we run now our tests again, it should work:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/package_spec.rb spec/classes/init_spec.rb

git::package
  install git-core
    should contain Package[git-core] with ensure => "latest"

git::init
  should contain Class[git::package]

Finished in 0.20456 seconds
2 examples, 0 failures

If you have problems with understanding the syntax of RSpec, just checkout the “The RSpec Book” by David Chelimsky.

Refactor

Since we now have green tests, we can play with the code. Let’s make manifests.init.pp nicer:

# manifests/init.pp

class git::init {
  # don't like the class declaration syntax: class { 'git::package': }
  include git::package # much better
}

Run the test:

$ rake spec
/home/helex/.rbenv/versions/1.9.2-p320/bin/ruby -S rspec spec/classes/package_spec.rb spec/classes/init_spec.rb

git::package
  install git-core
    should contain Package[git-core] with ensure => "latest"

git::init
  should contain Class[git::package]

Finished in 0.20158 seconds
2 examples, 0 failures

Your catalogue is still valid. Have beer because you have written your first tests for puppet and refactored your first manifest.

Conclusion

Testing is important – even or especially with the environment settings for your systems. It take some time to get used to it and you will find it in the beginning very cumbersome to write the code double. But when you are writing 4000 lines of code long manifest you will be happy to have structure, and confidence in your code with ya lovely tests.

Testing is good and saves your ass, especially the but of your company!

Checking your HDD

2012-07-12T00:00:00+02:00

Do one thing and do it well.
Unix Philosophy

This article describes a tool for testing your HDD on a daily basis and how can you fill it completely with zeros or random numbers to check it for broken sectors.

I was spending a whole day installing and configuring 4 different operating systems on my new Desktop PC – didn’t virtualize my Windows because I wanted to use it for gaming (especially Fallout 3). After four weeks the disasters happened: My hard disk drive was broken. Before spending a whole installing your new systems, it is worth spending a day to check your new HDD (Hard Disk Drive).

Symptoms of a breaking HDD

If you encounter any of the following things you can be sure that your HDD will break soon:

working with Blue Screens and occasionally system fall downs
problems with installing new OS – even on Knoppix
overwriting the harddisk completely
blinking LED even if you are not working – is a sign that the HDD disk is writing the content into still valid sectors

Checking your new HDD with smartools

Smartmontools is an analysis tool for Linux/Unix systems which allows you to check your hard disk – even on your regular usage. The program consists of two parts: smartctl (checking and evaluating HDD parameters) and smartd (is daemon to check your HDD on a regularly state).

Install the tool with the following command:

$ sudo apt-get install smartmontools

Using smartctl for HDD diagnosing

To get an overview of your new HDD please perform:

$ sudo smartctl -H /dev/sda

Important of the output is the parameter “passed”, it tells you that the test is over:

smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.0-23-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     ST500DM002-1BD142
Serial Number:    Z2ARNP4C
LU WWN Device Id: 5 000c50 04079c23c
Firmware Version: KC45
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Thu Jul 12 06:54:59 2012 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
          was completed without error.
          Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
          without error or no self-test has ever
          been run.
Total time to complete Offline
data collection:    (  600) seconds.
Offline data collection
capabilities:        (0x7b) SMART execute Offline immediate.
          Auto Offline data collection on/off support.
          Suspend Offline collection upon new
          command.
          Offline surface scan supported.
          Self-test supported.
          Conveyance Self-test supported.
          Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
          power-saving mode.
          Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
          General Purpose Logging supported.
Short self-test routine
recommended polling time:    (   1) minutes.
Extended self-test routine
recommended polling time:    (  81) minutes.
Conveyance self-test routine
recommended polling time:    (   2) minutes.
SCT capabilities:          (0x303f) SCT Status supported.
          SCT Error Recovery Control supported.
          SCT Feature Control supported.
          SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   107   099   006    Pre-fail  Always       -       12627760
  3 Spin_Up_Time            0x0003   100   100   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       76
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   064   060   030    Pre-fail  Always       -       2705616
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       84
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       74
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   061   049   045    Old_age   Always       -       39 (Min/Max 22/39)
194 Temperature_Celsius     0x0022   039   051   000    Old_age   Always       -       39 (0 21 0 0)
195 Hardware_ECC_Recovered  0x001a   045   039   000    Old_age   Always       -       12627760
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       110127256436820
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       3616779477
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       1189576021

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Conveyance offline  Completed without error       00%         2         -
# 2  Extended offline    Aborted by host               90%         2         -
# 3  Conveyance offline  Completed without error       00%         2         -
# 4  Short offline       Completed without error       00%         1         -
# 5  Extended offline    Aborted by host               40%         1         -
# 6  Short offline       Aborted by host               80%         1         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

You can perform longer tests of your harddisk with:

$ sudo smartctl -t long /dev/sda

Depending on the size of your HDD, it takes some time. For checking a whole 500 GB the program runs about 80 minutes.

It is even possible to check if your HDD has damage incurred during transporting the drive with the conveyance option:

$ sudo smartctl -t conveyance /dev/sda

Performing long time diagnostic with smartd

The first step is to give the daemon the permission to run checks in the background:

$ vim /etc/default/smartmontools
> start_smartd=yes

Find the line with and uncomment it:

# uncomment to start smartd on system startup¬
#start_smartd=yes¬

The configuration file for the daemon can be found under /etc/smartd.conf. Here only one line is sufficient to check all HDDs:

DEVICESCAN -m root@<your-os-name> -M exec /usr/share/smartmontools/smartd-runner

Let’s get through each line step-by-step:

DEVICESCAN: will scan all HDDs in the range between /dev/hd[a-I] .. /dev/sd[a-z], which support SMART
-m matthias.guenther@wikimatze.de: in case of an error, an email will be sent to this address
-M: the frequency of emails departure
- -M exec: don’t send testmails
- -M test: send a testmail (when using this option, you must leave out the /usr/share/smartmontools/smartd-runner)
- -M daily: send daily reports

Now it’s time to test our configuration:

$ smartd -q onecheck

You can check your internal mails with $ sudo mail (you need install Postfix on your OS) and rebooting the daemon:

/etc/init.d/smartmontools restart

If you want to have a graphical client for this tool, you need to run:

$ sudo apt-get install gsmartcontrol

You need then to start the program in sudo mode to detect all HDDs sudo gsmartcontrol.

Checking the whole space on your HDD

You know these tiny small dd commands? No, it’s time to learn and understand them. They are very handy to fill your your whole HDD with lovely zeros and ones.

dd stands for data definition. It has direct access on the harddisk and can read boot sectors – it is even used to create iso files of CDs.

Writing zeros or random numbers

$ dd if=/dev/zero of=/dev/sda

Take from the input file (if) the zeros (/dev/zero) and write them on the output file (of) /dev/sda. If the command has written the whole hard disk, it will end.

Speeding up the process with faster writing

Normally, each block of the hard disk has the size of 512 KB. To set the blocksize on 1 MB (1024 KB), we can speed up the writing speed with the factor two. We can achieve do this with the bs option.

$ dd if=/dev/zero of=/dev/sda bs=1024

Jump over errors

dd will end, if it detects a broken sector. With the conv=noerror option, dd will write till the HDD is full even over broken sectors:

$ dd if=/dev/zero conv=noerror of=/dev/sda bs=1024

Other useful parameters are notrunc (write the output file completely) or sync (write with the full length).

Running the dd command with status display

We can use the pv command to monitor the progress of data through a pipe. First we need to install it:

$ sudo apt-get install pv

To get an overview about how many MB or GB have already be written, use the following command:

$ dd if=/dev/random conv=notrunc,sync bs=1024 | pv > /dev/sda

Conclusion

A broken HDD is a bad thing, but if you know the symptoms it saves you a lot of time because you you know that you have to buy new hardware. Always have backup on an external HDD so that you can easily replace you data.

It is good to run a couple of tests before you start implementing your whole system.

Creation of a ruby gem

2012-06-07T00:00:00+02:00

Everything should be made as simple as possible, but not simple.
Albert Einstein

This article describes the crafting of a Ruby gem and decipher the mysterious GemSpec file.

The situation: For Jekyll I wanted to get an overview of how many pages, links, when the last build was, how many images I used in my blog, and when I lastly updated my bitbucket repositories. But before we dive deep into the creation, I will explain what a gem is and how you get started with writing your own gem. Don’t be afraid if you are writing a gem that already exists, your goal is to learn how to write a gem (and of course trying out new things).

Ruby Gem

A ruby gem is a self contained Ruby application which is packed as software. It can be downloaded and then used in other programs. Gems extends the core Ruby language through functions that are commonly used and may be of usage by other programmers. Many gems provides command line operations and help to automate tasks. Not all gems needs to contain a binary, like my sweetie gem. You can just through in the functionality provided by the gem to use it for your own programs.

Basic structure of a Gem

Below is a typical structure of gem:

.
├── README.md
├── Rakefile
├── lib
│   ├── sweetie
│   │   ├── bitbucket.rb
│   │   ├── conversion.rb
│   │   └── helper.rb
│   │   ...
│   └── sweetie.rb
│   ...
├── spec
│   ├── sweetie_bitbucket_spec.rb
│   └── sweetie_conversion_spec.rb
│   ...
└── sweetie.gemspec

A typical gem consists of the main class file (like sweetie.rb). This file contains other ruby-files (normally a list of require statements). The README.md briefly explains what the gem does, how to install it, explains the license, and should include small use cases for the gem (please checkout the README of the sweetie gem to see what I mean). The sweetie.gemspec file contains meta-information like who invented the gem, declare runtime environment dependencies, and so on. Additionally a gem should have test-files, so that other people can contribute to the Gem without damaging the main functionality.

Gemfile structure

First of all we create a Gemfile which defines important informations for the rubygems.org website. The site shows the author, the sources, the homepage, and some statistics (e.g. how often the gem was installed) – and by the way, it’s a nice place to sniff in the code of other hackers.

Here is an example sweetie.gemspec:

$:.push File.expand_path("../lib", __FILE__)
require 'sweetie/version'

Gem::Specification.new do |s|
  s.name             = 'sweetie'
  s.version          = Sweetie::VERSION
  s.date             = '2012-06-05'
  s.authors          = ['Matthias Guenther']
  s.email            = 'matthias.guenther@wikimatze.de'
  s.homepage         = 'https://github.com/matthias-guenther/sweetie'

  s.summary          = %q{Be short and precice!}
  s.description      = %q{Here you can write more and describe detailed features!}

  s.files            = `git ls-files`.split("\n")
  s.test_files       = `git ls-files -- {test,spec,features}/*`.split("\n")
  s.require_paths    = ["lib"]

  s.extra_rdoc_files = ["README.md"]

  s.add_runtime_dependency 'nokogiri', ">= 1.4.6"
  s.add_runtime_dependency 'json', ">= 1.6.1"
  s.add_development_dependency 'rake'
  s.add_development_dependency 'rspec'
  s.add_development_dependency 'yard'
end

Let’s explain the stuff which aren’t obvious:

$:.push: magic line that ensures “../lib” is in the load path
require 'sweetie/version': we need this line, because the module sweetie/version.rb contains the version number of the gem
s.files: uses a clever git command which lists all files under version-control – these files are directly included in the gem
s.test_files: again, a clever git listing command with ruby’s split() method – files that are used for testing the gem (the line supports TestUnit, MiniTest, Machinist, RSpec, Cucumber, and others
s.require_paths: these paths will be added to $LOAD_PATH when the gem is activated
s.summary: short summary of the gem’s description => will be displayed when running gem list -d
s.description: A long description of this gem => it goes more into detail than s.summary
s.extra_rdoc_files: here you can add files which can be used for the RubyDoc site – here the .gemspec README.md is the “main homepage” of the generated documentation
s.add_runtime_dependency: additional gems that will be installed when using this gem
s.add_development_dependency: additional gems needed when hacking on the gem (e.g. Rake is used for building and deploying the gem)

Building and Installing the Gem

Once you have created some code (it doesn’t matter how small the code is, except it must be valid and tested ruby code), you can (and should) build it on your local machine before making it public:

$ gem build sweetie.gemspec
=> Successfully built RubyGem
  Name: sweetie
  Version: 1.0.0
  File: sweetie-1.0.0.gem

Now you can install the gem local with the following command:

$ gem install sweetie-1.0.0.gem

And test the installation with:

$ gem list | grep sweetie
$ irb
>> require 'sweetie'
=> true

It’s working and you can experiment with the script locally.

Publishing your gem

The time has come to make your gem public – so that they can other hackers help. First, you need to register on rubygems.org. After that you can push the gem there with one simple command:

gem push sweetie-1.0.0.gem
Pushing gem to RubyGems.org...
Successfully registered gem: sweetie (1.0.0)

If you want to check if your gem is online, the console is your friend:

gem list -r sweetie

...

sweetie (1.0.0)

This will print all the gems which fit to the specified gem.

Conclusion

It is very easy to write an gem for ruby. Start small, create a repository on GitHub get something similar to hello world running, test your code with , and create a briefly documentation (read zach holman documentation talk to see why) about what the gem does, and give small examples. After this test your gem, deploy it on rubygems.org and spread the word about your accomplished on twitter – time to drink a beer!

Searching Haml files with ack

2012-05-14T00:00:00+02:00

This article describes how you can configure your ack settings to search for additionals files. UPATE: Andy Lester, the author of ack, give me feedback about this article.

Recently I was working on a Rails project and was using Vim with the :Ack <pattern> command to search after a certain typed string “Übreschrift” and would like to replace it with the correct version “Überschrift”. I typed into the console :Ack Übreschrift and I couldn’t find a single match. What went wrong?

Questions to answer

Why couldn’t I find the file with that content? Was I in the wrong directory? No:

$pwd
-> $HOME/git-repositories/brokenlifts

Did I used the wrong pattern? No, I searched after the right word “Übreschrift”. Did I used the wrong command? No, I tried ack-grep in the terminal to search for other common words like test or string and could only find results in .rb, .js files but not in *.html.haml files. Tada, found the mistake, ack is per default not configured to include these very special file types. If your press ack -f (thanks Andy Lester for this note) you will get the output of all files ack will go through – and this command didn’t displayed *.html.haml files.

Solve problems .ackrc

I created the $HOME/.ackrc and added the entries to include these special filename:

--type-add
html=.html.haml

Now my search worked and I got hits in *.html.haml files for searching the term “Übreschrift”.

Further refinements

Since I’m working with Rails there are other file types like sass, erb, less, scss, .. I would like to include into the search:

--type-add
html=.html.erb,.html.haml,.haml
--type-add
css=.sass,.less,.scss

And there might be directories I don’t want to have in my search path. Let’s ignore them and speed up our search:

--type-set
ignorables=.log,.tmp,.pdf
--ignore-dir=vendor
--ignore-dir=log
--ignore-dir=tmp
--ignore-dir=doc
--ignore-dir=coverage

If you would like to see the specified files for your grep environment search, just use ack --help types – this will print all information you need. Here is an example:

$ ack --help types
=>
  --[no]actionscript .as .mxml
  --[no]ada          .ada .adb .ads
  --[no]asm          .asm .s
  --[no]batch        .bat .cmd
  --[no]binary       Binary files, as defined by Perl's -B op (default: off)
  --[no]cc           .c .h .xs
  --[no]cfmx         .cfc .cfm .cfml
  --[no]cpp          .cpp .cc .cxx .m .hpp .hh .h .hxx
  --[no]csharp       .cs
  --[no]css          .css .sass .less .scss
  --[no]elisp        .el
  --[no]erlang       .erl .hrl
  --[no]fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp
  --[no]haskell      .hs .lhs
  --[no]hh           .h
  --[no]html         .htm .html .shtml .xhtml .html.erb .html.haml .haml
  --[no]ignorables   .log .tmp .pdf
  --[no]java         .java .properties
  --[no]js           .js
  --[no]jsp          .jsp .jspx .jhtm .jhtml
  --[no]lisp         .lisp .lsp
  --[no]lua          .lua
  --[no]make         Makefiles
  --[no]mason        .mas .mhtml .mpl .mtxt
  --[no]objc         .m .h
  --[no]objcpp       .mm .h
  --[no]ocaml        .ml .mli
  --[no]parrot       .pir .pasm .pmc .ops .pod .pg .tg
  --[no]perl         .pl .pm .pod .t
  --[no]php          .php .phpt .php3 .php4 .php5 .phtml
  --[no]plone        .pt .cpt .metadata .cpy .py
  --[no]python       .py
  --[no]rake         Rakefiles
  --[no]ruby         .rb .rhtml .rjs .rxml .erb .rake
  --[no]scala        .scala
  --[no]scheme       .scm .ss
  --[no]shell        .sh .bash .csh .tcsh .ksh .zsh
  --[no]skipped      Files, but not directories, normally skipped by ack (default: off)
  --[no]smalltalk    .st
  --[no]sql          .sql .ctl
  --[no]tcl          .tcl .itcl .itk
  --[no]tex          .tex .cls .sty
  --[no]text         Text files, as defined by Perl's -T op (default: off)
  --[no]tt           .tt .tt2 .ttml
  --[no]vb           .bas .cls .frm .ctl .vb .resx
  --[no]vim          .vim
  --[no]xml          .xml .dtd .xslt .ent
  --[no]yaml         .yaml .yml

Installing PostgreSQL gem under Ubuntu and Mac

2012-02-26T00:00:00+01:00

This article describes how to install the postgresql gem (called ‘pg’) under your local machine. The installation process is not trivial because you need to install the right packages: For Ubuntu/Linux you only need to install the right packages and for Mac you need to specify the correct path.

I checked out a new rails app called Broken lifts. Broken lifts in public transportation are annoying and limit the mobility of people in wheelchairs. The app provides status information from different operators on one page and shows how well they are performing.

Ones I checked out the project I have detected that it uses PostgreSQL as their database driver. So far I have never worked with it and I run into a couple of problems to get it running.

The problem

A small $gem install pg brought me the following error message:

$ gem install pg
Building native extensions.  This could take a while...
/Users/helex/.rbenv/versions/1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/ext/builder.rb:48: warning: Insecure world writable dir /Users/helex/bin in PATH, mode 040777
ERROR:  Error installing pg:
    ERROR: Failed to build gem native extension.

        /Users/helex/.rbenv/versions/1.9.2-p290/bin/ruby extconf.rb
checking for pg_config... no
No pg_config... trying anyway. If building fails, please try again with
 --with-pg-config=/path/to/pg_config
checking for libpq-fe.h... no
Can't find the 'libpq-fe.h header
\*\*\* extconf.rb failed \*\*\*
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
  --with-opt-dir
  --without-opt-dir
  --with-opt-include
  --without-opt-include=${opt-dir}/include
  --with-opt-lib
  --without-opt-lib=${opt-dir}/lib
  --with-make-prog
  --without-make-prog
  --srcdir=.
  --curdir
  --ruby=/Users/helex/.rbenv/versions/1.9.2-p290/bin/ruby
  --with-pg
  --without-pg
  --with-pg-dir
  --without-pg-dir
  --with-pg-include
  --without-pg-include=${pg-dir}/include
  --with-pg-lib
  --without-pg-lib=${pg-dir}/lib
  --with-pg-config
  --without-pg-config
  --with-pg_config
  --without-pg_config


Gem files will remain installed in /Users/helex/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/pg-0.13.2 for inspection.
Results logged to /Users/helex/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/pg-0.13.2/ext/gem_make.out

What relevant about the mess up there is the following part:

checking for pg_config... no
No pg_config... trying anyway. If building fails, please try again with
 --with-pg-config=/path/to/pg_config
checking for libpq-fe.h... no

Ruby can’t find the relevant setting of the PostgreSQL installation. Let’s change this!

Solution for Mac

install PostgreSQL library as described on the PostgreSQL page (opening the *.dmg file will install a bunch of stuff, so don’t be afraid of the amount of steps you have to perform)
check if you can find the installation open it under /Library/PostgreSQL/x.y – where x.y stands for the version you installed

Say you want to install the PostgreSQL manually with a gem install – you have to perform the following command

$ sudo PATH=$PATH:/Library/PostgreSQL/9.1/bin gem install pg

If you want to install it with Bundle, run the following command

$ sudo PATH=$PATH:/Library/PostgreSQL/9.1/bin bundle install

Please change the 9.1 with the version you have installed. Thats it!

Solution for Ubuntu/Linux

It is easier because you only have to install the right packages:

sudo apt-get install postgresql
sudo apt-get install libpq-dev

Now gem install pg should work.

Conclusion

It can be a really pain if you want to install gems and don’t have the right dependencies on your machine but if you initially invest some time on the problem, you have your enlightenment.

Compiling Vim from source for Ubuntu and Mac with ruby and python support

2012-02-19T00:00:00+01:00

This article describes how to build Vim (vim and gvim) from the source and compile it against a predefined version of ruby installed with rbenv. In the first chapter I’m describing how to build it from source for Ubuntu. The second part describes how to build for MacVim.

By installing Vim with ruby support from the sources, it is build against the system wide installation of ruby. If you already installed Vim and/or ruby with sudo apt-get install vim (or sudo apt-get install ruby) or with brew install vim (e.g. brew install ruby) if you are using OS X, remove it completely from your system to install the latest version of Vim.

Install rbenv

I’m installing rbenv on different machines so I created the following script (named rbenv_install.sh) to install ruby 1.9.2-p320:

cd $HOME
sudo rm -rf .rbenv
git clone git://github.com/sstephenson/rbenv.git .rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile

# install steps for rbenv-install command
cd $HOME/Downloads
sudo rm -rf ruby-build
git clone git://github.com/sstephenson/ruby-build.git
cd ruby-build
sudo bash install.sh

# updating the current shell
cd $HOME
exec $SHELL
source $HOME/.bash_profile

rbenv-install 1.9.2-p290
rbenv rehash
exec $SHELL

After this try the following:

$ rbenv global 1.9.2-p320
$ rbenv local 1.9.2-p320
$ ruby -v
=> ruby 1.9.2p320 (2012-04-20 revision 35421) [i686-linux]

The crowd applauds.

Get the latest version of Python

This will be put in the $HOME/lib folder:

$ mkdir $HOME/lib
$ cd $HOME/Downloads
$ wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
$ tar xjvf Python-2.7.3.tar.bz2
$ cd Python-2.7.3
$ ./configure --prefix=$HOME/lib
$ make && make install
$ make inclinstall # install headers, otherwise Vim won't have Python support
$ hash -r

Get the latest version of Vim

Visit vim.org and select the right download for your operation system (mainly Unix). If you are using a Unix system yo can get the latest Vim from here, download and unzip it:

$ cd $HOME/Downloads
$ wget ftp://ftp.vim.org/pub/vim/unix/vim-7.3.tar.bz2
$ tar -xjvf vim-7.3.tar.bz2

You can also get the latest Vim version from the git repository with the following command:

$ cd $HOME/Downloads
$ git clone https://github.com/b4winckler/vim
$ cd vim
$ git tag -l
$ git checkout v7-3-548

Compiling Vim and Gvim

To install Gvim on Ubuntu we need to install additional packages on our machine. The following snippets describe the packages for Ubuntu:

$ sudo apt-get install libncurses-dev libgnome2-dev \
 libgtk2.0-dev libatk1.0-dev libbonoboui2-dev libcairo2-dev \
 libx11-dev libxpm-dev libxt-dev

Since we are now having the Vim sources under $HOME/Downloads/vim it’s time to start the compilation. First we need to configure our compilation:

$ cd $HOME/Downloads/vim
$ ./configure --prefix=/usr/local \
    --enable-gui=no \
    --without-x \
    --disable-nls \
    --with-tlib=ncurses \
    --enable-multibyte \
    --enable-rubyinterp \
    --enable-pythoninterp \
    --with-python-config-dir=$HOME/lib/python2.7/config/ \
    --with-mac-arch=x86_64 \
    --with-features=huge \
    --enable-gui=gnome2

Let’s get over the heavy stuff:

--prefix=/usr/local – place of the binaries of the installed Vim installation (check the /usr/local/bin) – there will be the executable binaries
--enable-rubyinterp – says you want to build Vim with the default ruby installation (in our case /home/mg/.rbenv/shims/ruby)
--enable-gui=gnome2 – building Vim with Gvim support (if you don’t want Gvim than you can leave this line out)

After configuring the compilation check if the console response contains the following terms:

checking --with-ruby-command argument... defaulting to ruby
checking for ruby... (cached) /home/mg/.rbenv/shims/ruby
checking Ruby version... OK
checking Ruby header files... /home/mg/.rbenv/versions/1.9.2-p320/include/ruby-1.9.1

If you can’t see the lovely Ok, your Vim compilation will probably not have ruby support. After that we can build the configuration, install, and clean everything up:

$ cd ~/Downloads/vim
$ make
$ sudo make install
$ sudo make clean

Check the installation

Open a new session or perform exec $SHELL to reboot your Shell. You will see the fresh installed version of Vim:

$ which vim
/usr/local/bin/vim

$ which gvim
/usr/local/bin/gvim

Next check is to get the correct —version of vim and gvim with the following commands:

$ vim --version | ack ruby
$ vim --version | ack python
$ gvim --version | ack ruby
$ gvim --version | ack python

If both commands return +ruby and +python, you are fine, and got the achievement “I installed vim form source with ruby support on my own”. You should now be able to run the Hammer.vim plugin – install it, start it with :Hammer, install the missing gems and if you are able to run :Hammer without any missing dependencies, you have setup everything correct.

Installing MacVim from source

The ways are nearly the same as mentioned above: Install rbenv as mentioned in the steps before.

$ cd $HOME/Download
$ git clone git://github.com/b4winckler/macvim.git

Next step is to configure MacVim for OSX:

$ cd macvim/src
$ ./configure --prefix=/usr/local \
              --with-features=huge \
              --enable-rubyinterp \
              --enable-pythoninterp \
              --enable-perlinterp \
              --enable-cscope

After configuring the compilation check the console response after the following terms:

checking --with-ruby-command argument... defaulting to ruby
checking for ruby... /Users/helex/.rbenv/shims/ruby
checking Ruby version... OK
checking Ruby header files... /Users/helex/.rbenv/versions/1.9.2-p320/include/ruby-1.9.1

When this is finished, it is time build MacVim with make:

$ make

Next step is to run the MacVim installation:

open MacVim/build/Release/MacVim.app

The window should open MacVim. Run the following command in the MacVim quickfix window:

:ruby puts "MacVim"

In the next step, you can drop the MacVim.app icon in your Application folder and you are done.

Conclusion

It was a pain to gather the information for building vim by source with rbenv. Duration of finding that was: ~ 6 h. I’m not sure if all of this works on RVM – I just need additional feedback from people using it. Happy “Vim-ing”!

SMART principle

2012-01-12T00:00:00+01:00

This post is about the SMART principle that helps you to start and keep rolling projects. I outlined how I use SMART for my eBook project.

I recently read the book Pragmatic Programmer, in which Dave Thomas explained the SMART principle. SMART is an acronym and each character of the principle has the following meaning:

S (specific goal) – what is the goal of the new project you want to start
M (measurable) – list all the features you need to accomplish to start the project
A (action) – when will you start
R (realistic) – is the project small enough to finish it in the specified time frame demanded time frame
T (time) – in the beginning of July 2012 everything should be fine

Putting theory into practice

Late in 2011, I started a new and personally ambitious project: Writing an eBook about the great web framework Padrino. In a first step, I created a rough road map, about what I wanted to write and which topics should be covered. After writing the intro section and pushing the resources to github I did something very crazy and mentioned @padrinorb in one of my tweets and this was it — the authors of the framework motivated me with with writing the book. They retweeted every small progress I made with the project and through that I got further pull requests and even a bunch of interested followers who are watching the project.

Beginning of something great

Let’s see how I can formulate the Book project with SMART:

S (specific goal) – I want to learn how to write a eBook easily about Padrino web framework
M (measurable)
- create a roadmap about the topics of the book
- find a method to write book in one format and export the as pdf, epub, mobi, and HTML format
- get really involved in Open source community and communicate with them
- find an audience and get help from other people
A (action)
- start as soon as possible
- try to write everyday one chapter of the book
- tweet about the progress I made everyday
R (realistic)
- it is a big project (never have written a book before) but continuously delivery will motivate me during this long period
T (time)
- till February 2012 everything should be finished

I set up the frame of SMART goals and developed a rough plan – that’s all you need to let come your dreams true.

The SMART principle was first mentioned in 1981 as an issue of Management Review by George T. Doran.

Cost for SMART

To be honest, writing of my Padrino eBook project consumes a lot of my free time so I in the near future I will publish only a few new entries on this blog. I have many ideas and topics I want to cover, but first I will finish this book to deepen my writing experience. I’m glad to hear your opinion (or contribution) to the book – I will put every contributer on the special thanks list of the book.

Conclusion

SMART is great if you want to start in small steps to begin with something completely new, learn much about it, and report (for example, I’m using the twitter Hashtag #dailywriting) about the progress you made.

Thanks to Sebastian Guenther for reading an earlier draft of this post.

Global Coderetreat 2011 in Berlin

2011-12-04T00:00:00+01:00

I attended the Global Coderetreat event initialized by Corey Haines. The general purpose of this event was to get a bunch of hackers together to implement Conway’s Game of Life.

Setting up goals

As J. B. Rainsberger mentioned in his article four elements of simple design, it is best to setup goals for your retreat to get most out of it. Here is a list of what I achieved in each session:

Being a mentor of TDD: I taught dedan how to use TDD, why TDD helps you to think about your design.
Being a mentor of BDD: Showing a fellow Rubyist how to do BDD.
Doing something completely stupid: I played evil coder and ping-pong pairing with @tilosoenke – we couldn’t communicate with words, only with code, one writing tests and other the implementation of the tests.
Discuss about names: We said together and discussed about right names for methods, variables, and classes. We concluded that good names saves you scary comments in your code.
Explain Ruby to a Java-Programmer.
Being mentored about Lisp: We solved the whole ‘Game of Live’ in common Lisp) , while @HansHuebener explained me the strength and beauty Common Lisp.

Conclusion

It was great to get so many people with so many different skills and mindsets together, exchange experience and encouraging others to try something new. I will definitely have a closer look on Lisp to enhance my Ruby programming style.

Thanks to Sebastian Guenther for reading an earlier draft of this post.

Benefits of Scala

2011-10-29T00:00:00+02:00

When a new programming language is created, it came from a desire of one person or a group of to do it better than their forerunner. It’s an evolutionary process that humans are trying to improve themselves if something doesn’t as they want it to. Sometimes it’s only a small piece of improvement which creates an absolutely new feeling about something. By designing a new programming language you have a great pool of existing languages: LISP (1958), Smalltalk (1970), C++ (1979), Python (1991) and Ruby (1995). Today it is often told not to waste your time on building up an new language from the scratch. It’s time-consuming and why should you invent the wheel a second time? I think this is just normal. If you spend some time reading about the languages mentioned above, you will see that older language have still a great influence on new ones. Today, it is common to use DSLs (they are used extensively in Rails) to create new languages. Not everything happened in the past was bad and sometimes if you stick your head into some old forgotten technology you may find some diamonds.

This was the main idea of Odersky when he created Scala. It is the first language which is a real hybrid language and combines the invigorations of OOP and FP.

Main facts about Scala

Scala is ideal for today’s scalable, distributed, component-based applications that support concurrency and distribution.
Scala is statically typed that means that the type of some variable is immutable during the whole execution of the program.
There is modular mixin-composition for classes – some hack to enable multiple inheritance in Scala, which solves the diamond problem through linearity of the inheritance hierarchy via traits.
Support of so functions may have other functions as arguments which enables using anonymous functions.
Lower risk to use Scala in an existing Java Application because Scala works seamless with existing Java Code.
high level type system with variance annotations, compound types, lower and upper bounds for types
usage of inner and anonymous classes
implicit conversions – that means a function take one type as an argument and returns to another type (like converting an Integer into String)

Conclusion

Scala is a very rich language which combines many features of different languages. It’s great if you have to make different sections of your code cleaner, faster and conciser. See Scala at Linkedin to know what I mean.

Thanks to Hanna Schütt and Sebastian Gerth for reading an earlier draft of this post.

Mail Client in Java

2011-10-22T00:00:00+02:00

Most people spend more time and energy going around problems than trying to solve them.
Henry Ford

During the summer term 2007 we had the task to create a Thunderbird like mail client Java. The GUI should be created either with Swing or AWT. Beside the graphical implementation, we had to check and study the functionality of POP3 and SMTP. Only the correct numbers in this protocols are important because they stands for each transactions if it was a success or a failure. A first attempt this functionality was done with plain old sockets to get a feeling how POP3 and SMTP works.

After we a running function the basic functions of the mail client, we had to implement some extra-functions like attachments (mime parts in an email) or the handling of emails with drag-and-drop. I implemented this big project with the help of NetBeans.

Images

By means of the following images I will describe the basic layout of my email client.

The basic surface when the program starts. If you press on the upper task bar a drag-and-drop menu will open.

In this window you can add data of a new account.

Get the mail of an already existing account.

The window where to write a new email.

Drag-and-Drop window to manage emails.

Installation for Netbeans

get the sources from github
extract the javamail-1.4.zip => this package contains a Java library for handling emails
create a new project under NetBeans and copy all files of the Mail Client directory in the src directory of you NetBeans project
then you have to insert the mail.jar file in your project in the following way:
- click right on Libraries in the near of the view of your NetBeans project and chose Add JAR
- navigate to javamail-1.4 folder and select the mail.jar
- when you now select the files GUIMailsend.java and GUIMailsget.java should not show any errors
now you have to adopt the paths in the file GUIMain.java (static File f, String dir, FileReader file) and GUITree (File driveC, File source, File ziel) to your NetBeans project folder through absolute paths => I know it’s cumbersome but due to this date I couldn’t do any other
set the GuiMain.java as the main file in your project and start the program
the standard account is MG kontoinfos.kondat which contains the necessary data to access my spam account (you can guess how to change the data)
the directory Inbox MG saves all read mails of the user MG, further accounts have to be created each in an own directory
MailApi.java is not necessary for the mail client, this will just be used to check the correctness of the POP3 settings via the terminal
main cause of error messages:
- wrong paths to your project
- POP3 and SMTP
- mail.jar is not included correct in the project, but this will be shown in NetBeans

Conclusion

This project was my first big piece of software and I used all my gathered knowledge about Java. The classes are not bug-free, manually tested, has a size over 1000 LOCs, has absolute paths, and the function are sometimes very big. In the end it was just hacking down the software without using certain patterns or careful development with clean code. I tested my code with System.out.println and I lost so much time with this stupid testing. One good thing I did: I created comments and was able to get the program running even after two year.

Nowadays programming methodologies like Agile, Rapid-Prototyping or Model-Driven development together with the framework Rails let you develop my application with much more less effort and is more maintainable by driving the Test-Driven way, so you can refactor your code and don’t need to manual test part of your basis.

This project shows that it is possible to develop modular software in Java but in even greater projects it is easy to get lost in the lost in Java code if you don’t rely on a framework. It is even difficult to setup the project in different IDE – today you can use Vagrant to create an environment which can be deployed easily as Virtual images on different platforms.

Benefits of reading a book - and use it right after that

2011-10-15T00:00:00+02:00

We are what we repeatedly do. Excellence, then, is not an act, but a habit.
Aristotle

It’s a typical situation which every person knows: You look on your workspace and have a bunch of new books there. You ordered them because some great guys lay their hands in the fire that you need this wisdom. Of course you have to do a stressful job and don’t have time to read all the things but you still must learn new things in order to develop new skills. What should you do? I just started reading as much as I can.

What you should do during the process of reading

Don’t just read the book and put it away, read trough the table of contents, mark the parts which are interesting for you and draw a mind map. If you have questions which should be answered by the book, it’s good because then you will read the book with much more focus on them.

Don’t just think of reading the book like in school: Cram for a test about it, get a good grade and then just forget everything. This isn’t good and a waste of time. Don’t say you don’t have time, just think of allocating your time effective. For me it’s the best, just to use the new knowledge out of the book. This is the best way to get the most out of it. What you’re right using is just as good as cooking a new meal right after reading the recipe of it. You just inherit the procedure you recently learned with your own mind, own feelings, own observations and add your own style (think of your own ingredients in your favorite meal).

Benefits of note taking

Years after reading the book you will maybe come in the situation where you need some of the knowledge you just have read of this certain book you read it a lot of years ago. You know the title of the book but can’t remember anything. Before I explain the advantage of taking notes during reading a book, I suggest you my different methods for taking notes:

a pencil and a sheet of paper (which I personal prefer)
Netbook/iPad/cellphone (there you can directly take your notes)
you can use post-it or stick it to mark important passages
speaking notes of certain passages which you find very interesting
read an eBook with your kindle, iPhone or some other devices in which you can take custom-notes about something

The disadvantages of 1, 2 and 3 are carry the notes respectively the book with you (think you are on your trip to America and don’t want to fill your hole luggage with tons of books or do you want to travel without your favorite Simpsons trousers).

Method 4 is just a reminder of important passages. If such a file is very long you have problems in searching the needed sections – you have to play forward and back and this is boring. Don’t forget the amount of space which you need to store it. The advantage is, that you can easily take notes and train your language when your tongue is on air.

The usage of a specialized device for eBook-reading is a great opportunity. You can have many books on on your device, set bookmarks and can take notes. I think writing notes without a keyboard may have a worse usability – maybe with the iPad 2 its more easier. Of course, if you are used to this technology and you are are good then just use it. You can make it public or lock it with a certain password. But why hidden your knowledge so that no other one can see it? Let other benefit by your conclusions and wisdom.

Personally for me the best method is to write my impressions of a read book on a sheet of paper and transform with my PC later. A further step may be publishing your notes to your website – so you have the possibility to access your thoughts just via an Internet connection. I’m using a desktop-wiki zim and wrote a bunch of handy scripts, which transform the text directly in format of my web page.

Conclusion

If I had known early I would have started with this methods during my education at school. But better later then never. If I travel for a longer period and have a book with me, it has still some sheets of papers in it, so that I have space to take notes. A pencil is for me standard equipment. So I can put every place where I read a book my thought about it on paper. If the book is finished and I have a little bit time left, I will transform it into my Netbook and then have my thoughts with me.

So remember: In the beginning it’s just unusual to make several breaks during reading a book, which breaks your normal flow of reading. But during these breaks you can reorder your thoughts about certain things and get new insights. I suggest to publish it online and then see what will happen with it.

Thanks to Hanna Schütt and Sebastian Gerth for reading an earlier draft of this post.

Why learning a new programming language

2011-10-08T00:00:00+02:00

“Why should you start learning a new programming language” was the questions I asked me in the 2011? Nowadays we are all to busy to try new things, to expand our horizon. After reading the Pragmatic solving problems In the following chapter I will explain you, how I learned programming languages.

In the beginning there was Turbo Pascal

The first language I learned was Turbo Pascal. Everything (not only the programs) looked like a great “geek nerving god tool”. I quite don’t understand what I was doing with Turbo Pascal as my computer science teacher told me to make some simple calculation the perimeter for triangular and other geometric object. It was great to declare your variables and then put them into a function and print line the outcome. Here are some code snippets:

Turbo Pascal was my first procedural language and later during my studies I learned in a lecture about compiler construction that this language was developed 1970 by Niklaus Wirth (whom I admire for building such easy and effective techniques like the recursive descendant, but thats another tale). If you want to have this retro feeling just check out Ubuntu resources (or other things if you want to stay up).

The following images show the “IDE”. It has few options with fancy looking buttons and no options-overload.

Nice blue environment.

A successful compilation.

Syntax highlighting looks like in a old hacker movie.

Output in a console.

In secondary school we had written some programs in Delphi (original it had the name Object Pascal). The new things was that even the girls had little fun in writing programs in it (some nice “Hello Kitty” in with pink color, but they actually love it). We learned how to write functions and to create some appealing graphics. After that I lost my interests in programming because the age of warhammer or video games.

Then Java came …

When I started my studies of computer science I knew I had still no programming experience and during my first year I had to learn Java. It was quite good and quite bad if you ask my. The good things about it was that I had to learn an exhausting new methodologies: OOP, Interfaces, Recursion, Inheritance, Encapsulation, Polymorphism and many other concept. I was lost, I just wanted to learn how to write simple programs and no one could tell me, why I have to write this public static void main (String args[]) to start my program. My fellow students told my that I was an absolute newbie (they were actually right).

Chad Fowler wrote in his book The Passionate Programmer: “Always be the worst guy in every band you’re in.” I actually was it and in now time I made great progresses. But some aspects of Java hit my but: It was so long to express simple things, so much overhead with static, private and so on. To what I mean have a look on the following code:

class Socke {

  public String farbe;
  public int gewicht;
  private boolean istTrocken;

  void trockne() {
    istTrocken = true;
  }

  void wasche() {
    istTrocken = false;
  }

  boolean istTrocken() {
    return istTrocken;
  }
}

public class NewClass {

  public static void main(String args[]) {
    Socke stinki = new Socke();
    stinki.farbe = "schwarz";
    stinki.gewicht = 565;
    stinki.trockne();
    System.out.println(stinki.istTrocken());
  }
}

Ok, you may say: “It’s quite readably” but wait until you have written, about several kilo of LOCs and want to maintain or refactor it. I thought that I could understand my “well” written mail client but well afterwards you know it better.

For another term I looked on C++. It was good to see how pointer (or references) are working, how you must allocate your memory (Java has it’s ) and generics (or templates) for your data structures. I nearly learned something new, C++ is like Java, it’s OOP and you can program procedural if you like to. Have a look on some code:

/*
* 2.1
* Author: Matthias Guenther
* function: repeated shortening of breaches with a ggT
* use the Euklid-Algo to compute the ggT
*/

#include <iostream>
using namespace std;

int compute_ggT(int a, int b){
  if (b == 0){
    return a;
  } else {
    return compute_ggT(b, a % b);
  }
}

void shortening_breach(){
  int a, b, shortening_factor;
  string q;
  do {
    cout << "Please input the parameters to solve the breach: a/b\n";
    cout << "[Input] Parameter a: ";
    cin >> a;
    cout << "[Input] Parameter b: ";
    cin >> b;
    // look at the ggT
    if (b == 0){
      cout << "This is not allowed";
    } else {
      shortening_factor = compute_ggT(a,b);
      a = a / shortening_factor;
      b = b / shortening_factor;
      cout << "The breach is:" << a << "/" << b << "\n Continue?"
            "\n [Input \"q\" to leave the programm or \"c\" to "
            "continue]";
      cin >> q ;
      cout << "\n ";
      }
    } while (q != "q");
}

int main(){
  shortening_breach();
  return 0;
}

Enlightenment with Ruby

Normally I’m not the guy how runs after every hype but I haven’t found so far my beloved programming language. Why not risk a look. And then there was this ‘Whosh’ (think DC Comics). You could program in different styles, could design your own languages (called DSL) and this nifty framework Ruby on Rails (RoR) with Test-Driven Development (TDD), Behavior Driven Development (BDD), Continuous Integration (CI) and many more things. These things sound a like a mystery for me, but just get started and see how “easy” you can use it with RoR. It hits my head.

It was a very hard steady learning curve but it was more pleasantly then just go to the candy shop and get what you want for nothing. You have to invest much time. But look for yourself on ruby – it looks like a natural language. Just look at a script I wrote to tag MP3:

require "mp3info"

# setting the basedirectory
base_directory = "check/"

# run through all directories and get the mp3
Dir["#{base_directory}**/*.mp3"].each do |mp3|
  # get the directory
  arr_directory = mp3.split("/")[0..mp3.split("/").length-2]

  # empty string which will be merged to a string and which is needed for file renaming as a path
  directory_of_album = ""
  arr_directory.each do |part|
    directory_of_album << part + "/"
  end

  directory_of_album = directory_of_album.gsub("//", "/")
  mp3_track = ""

  # open the mp3 to read the id3 infos
  Mp3Info.open(mp3) do |track|
    # delete genre
    track.tag2.TCON = "Game"

    # check, if the track title exists
    if track.tag.title != nil
      # mp3 with an underscore are translated in / so we must catch this
      track_string = track.tag.title.gsub("/", "")
      track.tag.title = track_string
    else
      puts "Please set the track-tag for #{mp3.split("/").last} ... NOT converted"
    end
    # save mp3 tag
    mp3_track = track
  end

  # rename files
  if mp3_track.tag.title != nil
    mp3_newname = "#{Dir.pwd}/#{directory_of_album}#{mp3_track.tag.tracknum} - #{mp3_track.tag.title}.mp3"
    output = mp3_newname.split("/")
    File.rename(mp3, mp3_newname)
    puts "#{output[-1]} ... converted [#{mp3_track.tag.album}]"
  end
end

Automating tasks

I resisted very long to learn the bash, but there were so many tasks over the years which I didn’t do because t gives me the heebie-jeebies to do the same thing over and over again. After one weekend studying the open book “The Linux Command Line” by William E. Shotts I caught fire. For a long time I messed my desktop up with tons of images with ugly names, so I wrote a script to change this.

I have a bunch of images with ugly name, so I wrote a script to change this. Each command has it’s purpose you have to learn its syntax. Here is some sample code (pretty neat):

#!/bin/bash
z=`ls | wc -l`
z=0

for i in *.jpg; do z=$[$z+1]; mv "$i" ${i##*.jpg}$z\_$1.jpg; echo $i ; done # replace watermark_ (prefix)

It’s like Lego, you have these and that, plug it together to create something new. For my diploma thesis I create a Rakefile (the modern version of make written completely in ruby):

desc "Create a directory for the compiled classes"
task :create_class_directory do
  system 'mkdir -p classes' # overwrite the directory if it exist
end

desc "compile the plugin and put the files in the directory"
task :compile_plugin do
  system 'fsc -d classes Plugin.scala'
end

desc "copy the plugin descriptor"
task :copy_plugin_descriptor do
  system 'cp scalac-plugin.xml classes'
end

desc "create the jar file"
task :create_jar_file do
  system 'cd classes; jar cf ../divbyzero.jar .'
end

desc "create environment"
task :create_environment => [:create_class_directory, :compile_plugin, :copy_plugin_descriptor, :create_jar_file] do
  puts 'done with creating the jar file ...'
end

desc "compile the traits"
task :compile_traits do
  system 'fsc -d classes TraitProgrammingHelper.scala'
  system 'fsc -d classes TraitUpdaterHelper.scala'
  system 'fsc -d classes TraitArithmeticRulesHelper.scala'
  system 'fsc -d classes TraitBasicRulesHelper.scala'
end

desc "create environment with traits"
task :create_environment_complete => [:compile_traits, :create_environment]

desc "run example file"
task :test => [:create_environment_complete] do
  system 'scalac -Xplugin:divbyzero.jar Example.scala'
end

# ...

Head Scratching With Scala

I learned this language for my diploma-thesis, otherwise I still wouldn’t put a thumb on FP. So far I must say that it was a good decision to learn an functional language. Scala showed me how to use chained function, anonymous functions, currying, type bounds and many more constructs. In the beginning my head howls like a steam-tank, there was so much new terrain for me that I need several breaks to get used to it. After many hours of happy cramming I got surprisingly insights in doing OOP better. That was is, what the Pragmatic Programmers were preaching. To get new insight in a technology you have to regard it’s counterpart. Don’t hesitate to do something you would never do, you gain experience and more confidence in your profession.

Here is some of Scala code:

class Reference[T] {
  private var contents: T = _
  def set(value: T) { contents = value }
  def get: T = contents
}

object IntegerReference {
  def main(args: Array[String]) {
    val cell = new Reference[Int]
    cell.set(13)
    println("Reference contains the half of " + (cell.get * 2))
  }
}

Conclusion

This was my first essay. I tried to cover the things which whom I’m actually dealing. I presented my wide route from my programming experience so that you can see, how the things evolved for me. I always get my hands dirty and try to roll up my sleeves to get a touch about the newest technology. I think through adventuring you become each day a little bit better then yesterday.

Method alias in ruby

2011-10-01T00:00:00+02:00

In ruby you can create aliases for a method and variable name. This can be helpful if you want to override the behavior of some method without changing the origin implementation of it. alias_method take a new_name as a copy name of the old_name and it has the following syntax.

alias_attribute (new_name, old_name)

A small example:

class Davi
  def capital
    puts "Karaz-a-Karak"
  end

  alias_method :orig_capital, :capital

  def capital
    puts "Karaz-a-Karak rebuild"
    orig_capital
  end
end

davi = Davi.new
davi.capital

# output
"Karaz-a-Karak rebuild"
"Karaz-a-Karak"

What is the difference between `alias` and `alias_method`

alias is more general than alias_method and can be used to create an alias for global variable, regular expression backreference (like $&) or an existing method. Class variables, local variables, instance variables and constants may not be aliased.

def khemri_city
  puts "Nehekhara"
end

alias :orig_khemri_city :khemri_city

def khemri_city
  puts "Nehekhara new"
end

khemri_city

# output
"Nehekhara"
"Nehekhara new"

alias_method must return must be called on a method. So alias is more general than alias_method.

Conclusion

It is possible with alias_method to reopen a class, override a method call and you can still use the original call. In order to maintain backward compatibility alias_method are used in plugins, extensions, deprecating variables. Alias_method can be used in Rails to define action with duplicated content and remove duplicated code.

Here the duplicated variant:

class UsersController < ApplicationController
  def home
    list
  end

  def find
    list
  end

  def search
    list
  end
end

The DRY variant:

class UsersController < ApplicationController
  def home
    list
  end

  alias_method :find, :home
  alias_method :search, :home
end

Why I use Jekyll for blogging

2011-07-03T00:00:00+02:00

For several years I wante to write and tried many different platforms like wordpress blogger, or tumblr. The main problem which kept me away from writing was just the fact, that every time I want to write I just had to do it in a new environment and not in my favorite editor vim. Every system gives me the freedom to extend it in several ways but in the end it didn’t provide me the freedom to change every tiny piece I want. With jekyll I can use my favorite text editor and it really “turned me into a text monster”. This description sounds like a holy grail, but let me explain its abilities in the following sections.

What Jekyll is

Jekyll is a static site generator written in ruby. It generates static html pages. The page is presented through several templates and then fires the whole site, were articles are written in a text markup language like Textile or Markdown through the liquid converters to generate fully generated compiled website. Don’t think that it will be so easy for you to do it. First of all you have to learn either Textile or Markdown. I chose textile for writing my posts because I use Markdown to create the README files for my github account.

Setting up the environment

You need to have a valid ruby and ruby gems installation on your machine. A simple gem install jekyll will install the following gems:

directory_watcher: gives a list of files which change in some intervals
liquid: rendering templates in a safe manner
open4: creates a child process to handle pid, stdout, etc.
maruku: (Markdown interpreter
classifier: is a Bayes implementation and can be used semantic indexing like to display related post – this mechanism is used in machine learning)

To get nice syntax highlighting for your code you have to install pygments via sudo apt-get install python-pygments on Ubuntu/Debian. On the “install page(install link for jekyll install you can get more information about how to setup Jekyll.

The directories and styles

Here is the basic layout of a typical Jekyll project:

_includes: Small snippets which can be used in every place of the page.
_layouts: You can define layouts for post entries and the general default layout. Posts can have the special Yaml Front Matter.
_posts: Contains all posts in your specified markup language
_config.yml: Is a file to store configuration data like the styling of the URLs, or some global variables. It is also possible to define own variables which can be used as global things on other pages.

Other files can just put on the root directory like an atom.xml file (for RSS feed) or 404.html page. For example my post.html has the following layout:

---
layout: layout
---
<article class="post" role="main">
  <header>
    <section class="author">
    Posted by <a href="http://twitter.com/wikimatze" class="newwindow" title=""></a> on <time>Jul 03, 2011 </time>
    </section>
    <div class="clearer"></div>

  </header>
  
  <div class="clearer"></div>

  <footer>
  <section class="about-author">
  <h2>Discussion and Tweets</h2>
  <span class="about-author-image">
    <a href="https://twitter.com/wikimatze">
      <img src="http://en.gravatar.com/userimage/10525092/791987da4ba0ca49f1b8cc42bc7d4efb.png">
    </a>
  </span>
  <p>
    <strong>About the author:</strong> My name is Matthias Günther.
    This blog collects my thoughts about different topics as are my interests. I'm currently writing
    <a href="http://www.padrinorb.com/">Padrino book</a>, running <a href="http://www.vimberlin.de">vimberlin</a>, and being
    a <a href="http://railsgirlsberlin.de/">railsgirls-berlin</a> coach.

    <div class="clearer"></div>

    <div class="right">
  <a href="https://twitter.com/share" class="twitter-share-button" data-count="none" data-via="wikimatze">Tweet</a>
    <script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>
  <a href="http://twitter.com/wikimatze" class="twitter-follow-button" data-show-count="false">Follow wikimatze</a>
    <script src="http://platform.twitter.com/widgets.js" type="text/javascript"></script>
</div>


    <div class="clearer"></div>


    <div class="clearer"></div>

  </p>
</section>

  </footer>
</article>

The lines between --- mark a special Yaml Front Matter This block is treated as a special block in Jekyll and can contain different components. The `` stands for the content of a post entry.

Creating a layout

Here is the main template for my blog.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en-us">
<head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  
    <title>Why I use Jekyll for blogging</title>
  
  <meta name="author" content="" />
  <meta name="description" content="Writings, talks and pictures by . Günther works at MyHammer, loves painting Warhammer figures, and enjoys making cakes." />
  <link rel="alternate" type="application/rss+xml" href="" />
  <meta name="viewport" content="width=device-width, maximum-scale=1.0" />
  <meta name="robots" content="noodp, nodyr" />
  <link rel="stylesheet" href="/css/stylesheets/base.css" type="text/css" media="screen, projection" />
  <link rel="stylesheet" href="/css/stylesheets/syntax.css" type="text/css" />
  <link rel="stylesheet" href="/css/stylesheets/print.css" type="text/css" media="print" />
  <link rel="shortcut icon" href="http://farm8.staticflickr.com/7078/7284507972_a7aa341781_t.jpg" type="image/x-icon" />
  <link rel="canonical" href="/why-i-use-jekyll-for-blogging.html" />
  <link href="" rel="alternate" title="Blog of " type="application/atom+xml" />
</head>

<body>
  <div id="site" class="round">
    <a href="/why-i-use-jekyll-for-blogging.html#top" id="top"></a>

    <header id="page-header" role="banner">
      <span id="sitetitle">
        <a href="/index.html">wikimatze</a>
      </span>
      
      <nav id="navigation">
        <ul>
          <li><a href="/about.html">about</a></li>
          <li><a href="/projects.html">projects</a></li>
          <li><a href="/contact.html">contact</a></li>
          <li><a href="/follow.html">follow</a></li>
          <li><a href="/blog.html">blog</a></li>
          <li><a href="/talks.html">talks</a></li>
          <li><a href="/books.html">books</a></li>
        </ul>
      </nav>
    </header>

    <div class="clearer"></div>

    <div id="ribbon">
      <a href="https://github.com/matthias-guenther" rel="me" target="_blank">Fork me on Github</a>
    </div>
    <div class="seperator"></div>
    <div class="clearer"></div>

  
    <h1>Why I use Jekyll for blogging</h1>
  
    

    <footer id="page-footer" role="contentinfo">
      <nav>
        &copy;2012 
        &bull;
        <a rel="nofollow" href="/why-i-use-jekyll-for-blogging.html#top">top</a>
        &bull;
        <a rel="nofollow" href="">RSS</a>
        &bull;
        <a rel="nofollow" href="/imprint.html">imprint</a>
        &bull;
        <a rel="nofollow" href="/colophon.html">colophon</a>
        &bull;
        <a rel="nofollow" href="/donate.html">surprise me</a>
        <span id="last-build">last build: 5-7-2013</span>
      </nav>
    </footer>

  </div>

  <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.7/jquery.min.js"></script>

  <!-- Add fancyBox -->
  <link rel="stylesheet" href="/js/fancybox/jquery.fancybox.css?v=2.0.6" type="text/css" media="screen" />
  <script type="text/javascript" src="/js/fancybox/jquery.fancybox.pack.js?v=2.0.6"></script>

  <!-- Optionaly add button and/or thumbnail helpers -->
  <link rel="stylesheet" href="/js/fancybox/helpers/jquery.fancybox-buttons.css?v=2.0.6" type="text/css" media="screen" />
  <script type="text/javascript" src="/js/fancybox/helpers/jquery.fancybox-buttons.js?v=2.0.6"></script>
  <script type="text/javascript">
    $(document).ready(function() {
        $(".fancybox").fancybox(
          {
          wrapCSS    : 'fancybox-custom',
          closeClick : true,

          helpers : {
            title : {
              type : 'inside'
            },
            overlay : {
              css : {
                'background-color' : '#eee'
              }
            }
          }
        });
     });
   </script>
</body>

I’m using the meta-language Sass to create my CSS. There is one problem with Sass: You have to compile it every time you made a change. Fortunately, there is the compass which always compiles my sass file when I change it. When I build my page I started with compass watch css/ & a command to automate Sass building. This is very handy when changing the layout.

Conclusion

Just look on other pages what is possible with Jekyll. You can learn many new things by looking at other Jekyll blogs and copy what you need. I love to write a little bit and after I finished an article just perform rake deploy to upload my blog.

Matthias Günther

My Talk at Ruby Midwest 2013

Slides

Video

Impressions

Notes

Speaking at Ruby Midwest 2013

Getting started with RSpec Puppet

The tools

Setting up the environment

Testing, testing, and testing

Testing the creation of a package with a type

Refactor

Conclusion

Further reading

Checking your HDD

Symptoms of a breaking HDD

Checking your new HDD with smartools

Using smartctl for HDD diagnosing

Performing long time diagnostic with smartd

Checking the whole space on your HDD

Writing zeros or random numbers

Speeding up the process with faster writing

Jump over errors

Running the dd command with status display

Conclusion

Further reading

Creation of a ruby gem

Ruby Gem

Basic structure of a Gem

Gemfile structure

Building and Installing the Gem

Publishing your gem

Conclusion

Further reading

Searching Haml files with ack

Questions to answer

Solve problems .ackrc

Further refinements

Further reading

Installing PostgreSQL gem under Ubuntu and Mac

The problem

Solution for Mac

Solution for Ubuntu/Linux

Conclusion

Compiling Vim from source for Ubuntu and Mac with ruby and python support

Install rbenv

Get the latest version of Python

Get the latest version of Vim

Compiling Vim and Gvim

Check the installation

Installing MacVim from source

Conclusion

Further reading

SMART principle

Putting theory into practice

Cost for SMART

Conclusion

Global Coderetreat 2011 in Berlin

Setting up goals

Conclusion

Benefits of Scala

Main facts about Scala

Conclusion

Further reading

Mail Client in Java

Images

Installation for Netbeans

Conclusion

Benefits of reading a book - and use it right after that

What you should do during the process of reading

Benefits of note taking

Conclusion

Further reading

Why learning a new programming language

In the beginning there was Turbo Pascal

Then Java came …

Enlightenment with Ruby

Automating tasks

Head Scratching With Scala

What is the difference between `alias` and `alias_method`