TDD: Checking the return value of a Stub

State verification is used to ensure that after a method is run, the returned value of the SUT is as expected. Of course, you may need to use Stubs on a test double or a real object to tell the object to return a value in response to a given message.

In Java, you declare a method’s return type in its method declaration, this means that the type of the return value must match the declared return type or otherwise you will get a compiler error. In PHP, for example, you dynamically type the return value within the body of the method. This means that PHP mocking libraries cannot check the type of the return value and provide guarantees about what is being verified.

This leads to the awkward situation where a refactoring may change the SUT behaviour and leave a stub broken but with passing tests. For example, consider the following:

Developer (A) creates 2 classes, Presenter and Collaborator:

class Presenter
{
    protected $collaborator;

    public function __construct(Collaborator $obj)
    {
        $this->collaborator = $obj;
    }

    public function doSomething()
    {
        $limit = 1;
        $stories = $this->collaborator->getStories($limit);
        // ...
        return $stories;
    }
}

class Collaborator
{
    public function getStories($limit)
    {
        return array();
    }
}

Then writes a test case:

class PresenterTest extends PHPUnit_Framework_TestCase
{
    // Behaviour verification
    public function testBehaviour()
    {
        $mock = $this->getMock('Collaborator', array('getStories'));
        $mock->expects($this->once())
            ->method('getStories')
            ->with(
                $this->logicalAnd(
                    $this->equalTo(1), $this->isType('integer')
                )
            );

        $presenter = new Presenter($mock);
        $presenter->doSomething();
    }

    // State verification
    public function testState()
    {
        $stub = $this->getMock('Collaborator', array('getStories'));
        $stub->expects($this->once())
            ->method('getStories')
            ->will($this->returnValue(array()));

        $presenter = new Presenter($stub);
        $data = $presenter->doSomething();

        $this->assertEquals(array(), $data);
    }
}

The Developer (A) uses a mock to verify the behaviour (a mockist practitioner) and a stub to verify the method worked correctly. The first test asserts that the expectation is met and the second one that the given condition is true. Finally, the Developer runs and watches all of the tests pass. Great!

The next day Developer (B) decides to makes some changes to the Collaborator class and return NULL if there are no stories:

class Collaborator
{
    public function getStories($limit)
    {
        $stories = array();
        if (count($stories) < 1) {
            return;
        }

        return $stories;
    }
}

The implementation of the method-under-test changed, it now returns a different data type, null instead of array. This means that our second test should fail, but it doesn’t. The test still asserts that the given condition is true, even though the return type is different. This is a problem. It means that our second test is unable to verify the correct state of the SUT (and its collaborator).

This is because most PHP mocking libraries are heavily influenced by Java (PHPUnit was originally a port of JUnit), and Java doesn’t have this problem. In PHP, the method’s return type is not a required elements of a method declaration, so developers can define it at run time and return whatever type they want.

The solution

You can use DocBlock annotations to make sure the data type of the returned value matches the one defined in the DocBlock. For this to work you need to set the return value using ReturnValue instead of PHPUnit_Framework_MockObject_Stub_Return. For example:

class PresenterTest extends PHPUnit_Framework_TestCase
{
    // State verification
    public function testState()
    {
        $stub = $this->getMock('Collaborator', array('getStories'));
        $stub->expects($this->once())
            ->method('getStories')
            ->will(new ReturnValue(array()));

        $presenter = new Presenter($stub);
        $data = $presenter->doSomething();

        $this->assertEquals(array(), $data);
    }
}

Now if you run the test it fails with the following error message:

PHPUnit_Framework_Exception: Invalid method declaration; return type required

The test also fails if the returned type doesn’t match the expected one defined in the DocBlock:

class Collaborator
{
    /**
     * @return int
     */
    public function getStories($limit)
    {
        // ...
    }
}

Error message:

PHPUnit_Framework_Exception: array does not match expected type "int"

Or, if you specify more than one data type in the DocBlock:

class Collaborator
{
    /**
     * @return array|null
     */
    public function getStories($limit)
    {
        // ...
    }
}

Error message:

PHPUnit_Framework_Exception: getStories cannot return more than one type, 2 given (array, null)

This solution is not perfect but should work in most cases.

Installing multiple versions of Ruby using RVM

Ruby Version Manager (RVM) is a tool that allows you to install multiple versions of Ruby and have multiple versions of the same interpreter. Very handy for those who have to maintain different applications using different versions of Ruby.

To start, download RVM and install the latest stable version of Ruby:

$ echo insecure >> ~/.curlrc
$ curl -L https://get.rvm.io | bash -s stable --ruby
$ source ~/.bash_profile

Install an old version of Ruby:

$ rvm install 1.8.6
$ rvm use 1.8.6 --default
$ ruby -v
ruby 1.8.6

Create a Gem set and install an old version of Rails:

$ rvm gemset create rails123
$ gem install rails -v 1.2.3
$ rails -v
Rails 1.2.3

Switch back to your system:

$ rvm system
$ rails -v
Rails 2.3.5

Switch back to your RVM environment:

$ rvm 1.8.6@rails123

And, if you want to remove Rails 1.2.3, just delete the Gem set:

$ rvm gemset delete rails123

Alternatively to RVM, you also might look into rbenv.

Go is gaining momentum

Golang MascotThe Go language is gaining momentum among software engineers and PaaS/IaaS vendors. I think they all see potential in a simple, reliable, efficient and native-compiling language with a solid baseline API.

Thanks to its simplicity, performance, intuitive design and Google’s commitment to its future, Go could spark a change in the industry like we haven’t seen since the rise of Ruby in 2004. Ruby gained popularity among web developers and system administrators after David Hansson created Rails and Luke Kanies Puppet. Python, on the other hand, is still more attractive to teachers and academics and seems to be the preferred teaching language in many top universities, along with C/C++ and Java.

Go is more of an evolutionary than revolutionary language, and a great alternative to languages like Ruby, Python and C/C++, or platforms like Node.js. Experts predict that Go will become the dominant language for systems work in IaaS, Orchestration, and PaaS in the next couple of years. It has the potential to make a significant impact on server-side software, however, it still needs more adoption outside Google.

http://tour.golang.org/

API Development Tips

Organizations who are paying attention already know they need to have an open web API, and many already have under development or in the wild. Make sure you haven’t been caught by the pitfalls of many early API releases.

Multiple points of failure

  • Back-end systems: db servers/caches, hardware failures, etc.
  • Interconnections: router failures, bad cables, etc.
  • External Dependencies: fail whales, random cloud latency, etc.

The 5 tips

  1. Test it all
    • Unit test are not enough, they are just the beginning.
    • Test what users experience. Perform end-to-end black box tests.
    • Replay your access logs. Very accurate.
    • Validate return payloads. A stack trace is not valid XML.
  2. Plan for future versions
    • Versions are not sexy/semantic (but do it anyway).
    • Announce versions often.
  3. Embrace standards
    • APIs are better when predictable.
    • Standard approaches mean tools.
    • Avoid uncomfortable migrations. No one wants an OAuthpocalypse.
  4. Monitor everything & be honest
    • Trends are your friend.
    • Users are not your early-warning ops team.
    • Be open and honest, or your users will tweet that your API sucks!
  5. Fail well
    • Well-formed errors win friends and makes users more tolerant to failure.
    • Make monitoring easy.
    • Don’t punish everyone. Determine who gets hurt most by failures.

Watch the video here: Understanding API Activity by Clay Loveless

Intercepting class method invocations using metaclass programming in Python

In Ruby, objects have a handy method called method_missing which allows one to handle method calls for methods that have not been defined. Most examples out there explain how to implement this in Python using __getattr__, however, none of them (honestly, none) explain how to intercept class method (@classmethod) invocations using __metaclass__.

And that’s why I wrote this post.

The function type is the built-in metaclass Python uses, it not only lets you know the type of an object, but also create classes on the fly. When you write, for example: class Example(object) the class object Example is not created in memory straight away. Python looks for the __metaclass__ attribute in the class definition and if it finds it, it uses it to create the object class Example. If it doesn’t, it uses type to create the class. The main purpose of a metaclass is to change the class automatically, when it’s created.

Here’s an example of how to use metaclass programming to intercept class method calls similar to the method_missing technique in Ruby:

class ClassMethodInterceptor(type):

    def __getattr__(cls, name):
        return lambda *args, **kwargs: \
                   cls.static_method_missing(name, *args, **kwargs)

    def static_method_missing(cls, method_name, *args, **kwargs):
        e = "type object 'static.%s' has no attribute '%s'" \
            % (cls.__name__, method_name)
        raise AttributeError(e)

class Example(object):

    __metaclass__ = ClassMethodInterceptor

    def __getattr__(self, name):
        return lambda *args, **kwargs: \
                   self.method_missing(name, *args, **kwargs)

    def method_missing(self, method_name, *args, **kwargs):
        e = "type object '%s' has no attribute '%s'" \
            % (self.__class__.__name__, method_name)
        raise AttributeError(e)

    @classmethod
    def static(cls):
        print 'static.%s' % cls.__name__

    def instance(self):
        print self.__class__.__name__

Console:

>>> Example.static()
static.Example
>>> Example.foo()
Traceback (most recent call last):
...
  File "example.py", line 12, in static_method_missing
    raise AttributeError(e)
AttributeError: type object 'static.Example' has no attribute 'foo'
>>> e = Example()
>>> e.instance()
Example
>>> e.foo()
Traceback (most recent call last):
...
  File "example.py", line 26, in method_missing
    raise AttributeError(e)

AttributeError: type object 'Example' has no attribute 'foo'

If you ever implement something like this, remember that Python doesn’t distinguish between methods and attributes the way Ruby does. There is no difference in Python between properties and methods. A method is just a property whose type is instancemethod.

Check whether your web server is correctly configured

Last year Zone-H reported a record number of 1.5 million websites defacements. 1 million of those websites where running Apache.

When it comes to configuring a web server, some people tend to turn everything on by default. Developers are happy because the functionality that they wanted is available without any extra configuration, and there is a reduction in support calls due to functionality not working out-of-the-box. This has proven to be a major source of problems for security in general. A web server should start off with total restriction and then access rights should be applied appropriately.

You can check whether your web server is correctly configured by using Nikto, a great open source vulnerability scanners that is able to scan for quite a large number of web server vulnerabilities. From their site:

“Nikto is an Open Source (GPL) web server scanner which performs comprehensive tests against web servers for multiple items, including over 6400 potentially dangerous files/CGIs, checks for outdated versions of over 1200 servers, and version specific problems on over 270 servers. It also checks for server configuration items such as the presence of multiple index files, HTTP server options, and will attempt to identify installed web servers and software. Scan items and plugins are frequently updated and can be automatically updated.”

I’m going to run a default scan by just supplying the IP of the target:

$ cd nikto-2.1.4
$ ./nikto.pl -h 127.0.0.1

- ***** SSL support not available (see docs for SSL install) *****
- Nikto v2.1.4
---------------------------------------------------------------------------
+ Target IP:          127.0.0.1
+ Target Hostname:    localhost.localdomain
+ Target Port:        80
+ Start Time:         2011-12-12 13:06:59
---------------------------------------------------------------------------
+ Server: Apache
+ No CGI Directories found (use '-C all' to force check all possible dirs)
+ 6448 items checked: 0 error(s) and 0 item(s) reported on remote host
+ End Time:           2011-12-12 13:08:07 (68 seconds)
---------------------------------------------------------------------------
+ 1 host(s) tested

By looking at the last section of the Nikto report, I can see that there are no issues that need to be addressed.

Tools like Nikto and Skipfish serve as a foundation for professional web application security assessments. Remember, the more tools you use, the better.

Links

JavaScript: Retrieve and paginate JSON-encoded data

I’ve created a jQuery plugin that allows you to retrieve a large data set in JSON format from a server script and load the data into a list or table with client side pagination enabled. To use this plugin you need to:

Include jquery.min.js and jquery.paginate.min.js in your document:

<script type="text/javascript" src="js/jquery.min.js"></script>
<script type="text/javascript" src="js/jquery.paginate.min.js"></script>

Include a small css to skin the navigation links:

<style type="text/css">
a.disabled {
    text-decoration: none;
    color: black;
    cursor: default;
}
</style>

Define an ID on the element you want to paginate, for example: “listitems”. If you have a more than 10 child elements and you want to avoid displaying them before the javascript is executed, you can set the element as hidden by default:

<ul id="listitems" style="display:none"></ul>

Place a div in the place you want to display the navigation links:

<div id="listitems-pagination" style="display:none">
    <a id="listitems-previous" href="#" class="disabled">&laquo; Previous</a>
    <a id="listitems-next" href="#">Next &raquo;</a>
</div>

Finally, include an initialization script at the bottom of your page like this:

<script type="text/javascript">
$(document).ready(function() {
    $.getJSON('data.json', function(data) {
        var items = [];
        $.each(data.items, function(i, item) {
            items.push('<li>' + item + '</li>');
        });
        $('#listitems').append(items.join(''));
        $('#listitems').paginate({itemsPerPage: 5});
    });
});
</script>

You can fork the code on GitHub or download it.