Debugging web applications - backend

Backend debugging can be tedious because the applications is sometimes running on different computer than you are writing it. Install local PHP interpreter, web server and database for serious development. Unfortunately, the error can sometimes be caused by the “environment”, i.e. the code works at your local server and does not work in production. This can be causes by installed libraries, related applications like database or Apache web server and operating system (type and version).

PHP debugging

PHP has quite nice error reporting mechanism. It tells you the line number and the file where the error occurred. Following code produces quite handy error message.

<?php
$sum = $a + $b; //these two variables are not defined
echo $sum;

PHP error

There is one thing to be aware of though: when you miss a semicolon, the error will be reported on the following line.

line  9:    //...
line 10:    $stmt = $this->db->query('...')              //no semicolon here
line 12:    //empty
line 13:    $tplVars['something'] = $stmt->fetchAll();   //the error will be reported for line 13
line 14:    //... 

PHP has errors divided into several levels (from ERROR through WARNING to NOTICE). Some errors cause your code to halt (e.g. PARSE errors), others like WARNING or NOTICE are just informational – your code keeps running. Notice that the first example still printed the resulting value “0” after those two errors. You can even disable error reporting with a function.

Direct error reporting is usually disabled on production servers. Therefore these servers are not good for development because you can miss out some problems – e.g. PHP made the values of the variables $a and $b zero in the first example – it is nice, but you should define the initial values of variables by yourself. You never know, where you application will run and whether the administrator of the server will enable the error reporting in the future. Serious applications override the error reporting setting to report everything but they also set an error handler which puts all error reports into a log file and not onto a screen of a visitor.

There are also exceptions which are thrown by some functions. Unhandled exceptions also causes your applications to stop – always use try-catch block when you know that a function can throw exception. And you can also set a handler for unhandled exceptions.

Stack trace

Sometimes you can encounter a stack trace. Usually when you use some kind of framework with enabled debugging. At first it might seem very odd and quite bulky. Stack trace is a printout of executed functions from top to bottom. It shows the path of the interpreter in the code, including the code of the framework (which is not very useful). Every line in the stack trace is a function call which brought the interpreter closer to the error. Take a look at following image of Slim framework generated stack trace:

Error stack trace

Different frameworks have different stack trace printout tools. Some of them are more useful, some of them less. Do not be intimidated by the amount of information and try to find your PHP files in the stack trace and determine the problem. Read the error message too.

The stack trace is often caused by uncaught exception, always use try-catch around SQL queries and disable debugging mode of the framework in production (use logs).

Concrete error examples

Now that I told you something about PHP’s error reporting system, I can proceed to concrete examples. There is usually two types of actions in the backend: retrieval and display of information and processing of inputs.

You have to be aware of all the kinds of possible PHP script inputs and conditions of execution, because the combination of input parameter values and environment conditions is crucial to trigger errors:

  • query parameters (e.g. /some-route?id=123)
  • post data (from <form method="post">)
  • route placeholders (e.g. /some-route/123)
  • session values
  • HTTP headers
  • cookies (passed via headers)
  • data in the database
  • PHP config
  • operating system and installed software and libraries (Linux VS Windows, Apache version, file system)
  • version of PHP itself and installed plugins
  • version and type of database (MySQL VS PostgreSQL VS SQLite VS …)

Easiest approach to determine content of a variable is to stop PHP script just anywhere and display contents of variables using echo or print_r() function and then calling exit. Another way is to use logger.

You might seen using breakpoints and stepping through your code in another programming environments or just read about it in JavaScript section. It is not impossible to do this with PHP, but it is problematic because your code is not executed through the IDE. PHP is executed in a web server’s context (and that also can be on a different machine). To be able to set breakpoints you need to install PHP debugger like Xdebug and connect it with your IDE. This is very difficult for beginners.

Input processing errors

Input processing bugs are often caused by typos in variable naming – check name attribute for each input element and array keys used to access request parameters (either query or body). Input processing is a bit more complicated, because the error can actually be caused by previous rendering of another route.

<form action="/errorneous-route" method="post">
    <input type="text" name="firstName">    <!-- the error is either here... -->
    <input type="submit" value="OK">
</form>
$app->post('/errorneous-route', function(Request $request, Response $response) {
    $data = $request->getParsedBody();
    
    print_r($data);     //is it really there?
    exit;               //stop execution
    
    if(!empty($data['first_name'])) {   //...or here 
        //do some work
    } else {
        exit('Supply first name value.');
    }
});

A common problem with form-processing debugging is that you redirect the user away from the POST route. The easiest thing you can do is to comment out the redirect command or use the exit command to stop PHP script.

$app->post('/errorneous-route', function(Request $request, Response $response) {
    try {
        //...
        
        echo $someValue;    //print some value
        exit;               //stop execution
        
        return $response->withHeader('Location', $this->router->pathFor('other-route'));
    } catch(Exception $e) {
        $this->logger->error($e->getMessage());
        exit('DB error');
    }
});

HTTP protocol debugging

Use developer tools (usually under F12 key) to display contents of GET, POST or cookie request parameters. Following image shows Chrome developer tools with network console opened. You can find posted values (red) and cookie values (green) in the details of the request. You should check the values and keys in HTTP request before you start debugging PHP script.

POST variables in chrome developer tools

Another thing to check, although not that important for classical web applications, is HTTP status code of the response. This is much more important for AJAX applications.

Firefox has very similar tools for this.

Data retrieval errors

Data retrieval is the process which extracts data from the database (using SQL query) and displays them to the user. This part is more straight forward because everything is done in single load of the page. If your script depends on query/post parameters, refer to previous section to check the incoming data.

Bugs related to retrieval and presentation of information are often caused by wrong SQL queries. Stop your code right after the query is executed and print the result to be sure what came out of the database.

$app->get('/errorneous-route', function(Request $request, Response $response) {
    try {
        //the problem is using wrong operator to compare against NULL (should be IS NULL)
        $stmt = $this->db->query('SELECT * FROM person WHERE id_location = NULL ORDER BY last_name');
        
        $tplVars['persons'] = $stmt->fetchAll();
        
        print_r($tplVars['persons']);     //print the result of query, you should see none
        exit;                             //stop the execution
        
        return $this->view->render($response, 'tpl.latte', $tplVars);        
    } catch(Exception $e) {
        $this->logger->error($e->getMessage());
        exit('DB error');
    }
});

It is crucial to use correct format for given data-types. For example DATE columns need the value to be passed as YYYY-MM-DD. You have to convert any national date format into that. Always set value for columns without defined default value although some SQL systems use pseudo-default values (e.g. empty string for VARCHAR column). To pass NULL, use real PHP’s null constant, not '' (empty string) or 0 (zero). Remember that everything that came from HTTP request as parameter is a string (yeah, because HTTP is text based protocol).

$app->post('/post-route', function(Request $request, Response $response) {
    $data = $request->getParsedBody();
    $val = !empty($data['...']) ? $data['...'] : null;
    $stmt = $this->db->prepare('INSERT ...');
    $stmt->bindValue(':col', $val);
    //...
});

SQL query debugging

Using PDO’s prepared statements is great for security but very bad for debugging. There is actually no way to see the a query with placeholders replaced with actual values. It is because the query and parameters are passed into the database system separately and the replacement of placeholders is actually carried out by the database engine itself.

To be sure about the query result use Adminer or another tool to build the query before you plug it into your source code.

Another error-rich place is the interface between template and PHP code. Make sure that you understand the structure that you pass in $tplVars variable to the templating engine and that you use right variable names in the template.

Problem with template engine lies in the fact, that template files are not executed as they are, they are in fact converted into real PHP code (which is stored in cache) and therefore the PHP error reporting from template is very confusing (it points to some cached files). PHP code which represents the template is usually loosely similar to the template, you can open the compiled template from cache and look for the line which reported the error to determine the actual problem in the template.

Template error

Let’s see what is around line 68 in that file:

<?php
  $iterations = 0;
  foreach ($persons as $person) {
?>
  <tr>
    <td><?php echo LR\Filters::escapeHtmlText($person['first_name']) /* line 15 */ ?></td>
    <td><?php echo LR\Filters::escapeHtmlText($person['last_name']) /* line 16 */ ?></td>
    <!-- following line is number 68 in compiled template -->
    <td><?php echo LR\Filters::escapeHtmlText($person['nicknam']) /* line 17 */ ?></td>
    <td><a href="<?php
		  echo $router->pathFor("addContact");
	    ?>?id=<?php echo LR\Filters::escapeHtmlAttr(LR\Filters::safeUrl($person['id_person'])) /* line 18 */ ?>">Add contact</a></td>
  </tr>
<?php
    $iterations++;
  }
?>

Latte templating engine even includes original line numbers as comments on some of the lines. You can look for similar structure in the original template file:

{foreach $persons as $person}
<tr>
  <td>{$person['first_name']}</td>
  <td>{$person['last_name']}</td>
  <!-- following line is number 17 in latte file -->
  <td>{$person['nicknam']}</td>
  <td><a href="{link addContact}?id={$person['id_person']}">Add contact</a></td>
</tr>
{/foreach}

Those two files are not that different, you can easily see, that the problem was a typo in array key nickname. My example was a very simple error, many errors are hard to find because the template is messy – use indentation, good editors have functions to format code automatically, and good editors also have macro pairs highlighting. Some templates are very long – use inheritance and includes to avoid code repetition. Sometimes the problems is not with template syntax, but with rendered HTML structure, this is covered in frontend debugging article.

Template syntax errors

Another type of issue related to templates is error in the syntax of template itself. Sometimes the error report is somewhat useful, in this case, it at least says which file is wrong (but not the line).

Template syntax error

The cause of this error is missing {/if} block:

{extends layout.latte}
{block body}
  <!-- some html -->
  {if $message}
     <!-- nested html -->
  <!-- missing {/if} -->
  <!-- more html -->
{/block} <!-- line 24 -->

In some cases, the template error report is useless or the template itself is too complicated and it is hard to spot the error. Try to remove pieces of template code until the page starts rendering something. This way you can locate the problem and focus on that part of code. Following error report shows Latte error without specified template file (the error is raised from FileLoader.php which is part of Latte engine).

Template syntax error

In this case, the error is in {include ...} macro which was specified without the file name or block name parameter.

Sometimes the application works correctly on your machine, but runs into difficulties on the production or testing server.

PHP version

Newer frameworks or libraries require PHP 7 which added many new functions. Check the production environment, if you have no control over it, you have to adapt.

Missing libraries or misconfiguration

PHP uses system of plugins, sometimes the set of installed plugins on other machine is different that yours. Use phpinfo() function to check installed plugins and their config. This function also shows important information about maximum execution time or script memory limit.

Permission denied

A very common error is the “Permission denied”, it occurs when your application tries to write some data to a disk. The problem is, that the user who uploaded the script (i.e. you) is not the same user who executes the script to generate HTTP response (user who runs Apache web server). You have to allow that other user to write into your folders and files. Therefore when you plan to write files on disk (logs, cached templates, uploaded images etc.), you have to set write permission for others (e.g. chmod 0777) for that folder/file. You can read how to do this in technical support section.

Permission denied

Summary

In comparison with frontend, the backend environment is much more determinate. You usually have only one server with given versions of PHP, Apache and a database. A great problem is when everything works in development environment and fails in production.

New Concepts and Terms

  • error levels
  • error handler
  • exception handler