whateverthing.com

☰

Occasionally at work I'm presented with a bug that's a bit of a stumper. The kind of bug that defies rational explanation, that you can only solve by exploring far more deeply into the guts of your tech stack than you would like to.

This is the story of one such bug.

As a bit of background, I've been working on the second version of our product's RESTful-ish API. When we built the API, we took the approach of exposing it as a thin wrapper over existing endpoints that were undocumented and mostly intended just for application functionality. Incoming API requests get packaged up and internally dispatched to those existing endpoints.

When testing one of these API-level endpoints, a coworker noticed that it did not appear to be respecting the limit parameter. The documentation I had written said that the default limit should be 1000 entities returned for each request. The coworker was observing numbers in excess of 1000 when he didn't provide a limit parameter, which shouldn't have been happening.

I ran my own test, and to my surprise, I received a maximum of 25 entities.

After looking through the source, I determined that my documentation's suggestion of 1000 entities by default wasn't being enforced at the API level, and instead the existing endpoint's limit of 25 entities was taking priority.

However, this did not explain why my own test returned 25 entities, while my coworker's test essentially returned unlimited entities.

I got him to use file_put_contents() to try and log the value of the limit parameter as it was being passed through the system. It appeared to be null, which was odd because the existing endpoint's default of 25 was supposed to override null values.

Clearly, this was an unexpected situation and required further research. I was able to determine that he was testing on CentOS 6.x with the bleeding-edge PHP version of 5.3.3 (Seriously, if you are running a PHP version that old, DO EVERYTHING IN YOUR POWER to get onto a supported version such as 5.5 or newer) - because this is one of our product's supported PHP versions, we still had to get to the bottom of the issue.

I grabbed an ISO of CentOS and went about getting it set up on my local machine, to see if I could observe the problem first-hand.

As soon as I got it up and running, I confirmed that it was behaving exactly as described. Since my main machine was running on PHP 5.5, and the virtual machine was running on PHP 5.3.3, it seemed reasonable to conclude that there was some difference between those PHP versions.

It was up to me to isolate and identify that difference.

I traced through the source of our framework (Zend Framework 2) while puzzling over what would be causing the existing endpoint's behaviour to vary between PHP versions.

Here's how the API endpoint fetched the limit parameter:

$request->getQuery('limit');

Here's how the existing endpoint fetched the limit parameter, specifying 25 as the default parameter:

$request->getQuery('limit', 25);

On the surface, the code should always be defaulting to 25. However, once I dug a little deeper, I started to see what the issue was.

In the Request::getQuery() method, Zend used the $name parameter (in this case containing the string limit) as an ArrayObject key. When it came time to detect if the parameter had been provided in the request, Zend called isset($this[$name]), and if that returned false, Zend returned the provided default value.

Well, when the API endpoint dispatched an internal request to the existing endpoint, it was always specifying a limit parameter in the array of query values. Even if the value for it was null.

On some versions of PHP, the isset($this[$name]) call in an ArrayObject will see this null and be like "Nope! That's not set. That's not a real value. Skipping that one. Here's your default."

PHP 5.3.3 is not one of those versions of PHP.

In fact, once I narrowed down the cause, I was able to use 3v4l.org to replicate the problem and isolate which PHP versions were affected.

<?php

class Params extends ArrayObject {
    public function __construct(array $values = null) {
        if (null === $values) { $values = array(); }
        parent::__construct($values, ArrayObject::ARRAY_AS_PROPS);
    }

    public function get($name, $default = null) {
        if (isset($this[$name])) {
            return $this[$name];
        }
        return $default;
    }
}

$data   = array('limit' => null, 'test' => 20);
$params = new Params($data);
echo 'Expected: 25, actual: ' . $params->get('limit', 25);

The 3v4l.org service reported that PHP 5.1.0-5.3.10 and PHP 5.4.0 were affected.

The solution was to enforce the limit default in the API endpoint, as should have been done in the first place - but arriving at that solution took quite an odd journey.

---

It should be noted that newer versions of Zend Framework 2 alter the way the Zend ArrayObject class checks for existence of query parameters. It appears that these newer versions have standardized on a behaviour similar to PHP 5.3.3 - possibly treating null as a valid value. I haven't tested to see if this is the case.