Using reStructuredText to generate an ERD

Recently I had to create an ERD for a project, and I didn’t wanted to use a binary (and mostly a proprietary) format. The reasons for this is that I don’t want to bind myself to a specific program, that I want to be able to use the data in different places/formats, and most importantly that I can version control it (and diff it as text).

Next to Visio, Google Draw, Omnigraffle, etc… there are new players like Skipper, that can be used to easily generate Doctrine models, but these are still a bit to limited for me.

Because I like to be verbose in decisions I make when designing models and how they fit together, I usually start with a basic text file listing entities and there properties. Those text files evolved to MarkDown files, and recently I started using reStuctured text (.rst) files, because it is a very powerfull format (and it has a lot of parsers, which will help me achieve my goals I stated in the beginning of this blog post).

A simple file could start like this:

Entity-Relationship Diagram
===========================

This file is used to describe all entities, their properties and how they relate.

User
----

A user entity is used to...

Fields
++++++

:username: The username a user will use to login to his/her account
:email: ...

Group
-----

A group of users can have specific roles in the application.

Fields
++++++

:name: The group name.
       This should be...
:roles: An list of roles

Using this format, we can add a lot of background information and annotations, while keeping a structured format we can use later.

A lot of viewers (GitHub included) can present the reStructuredText quite good already, including references in the document. If your document gets quite large, the link can be very handy:

...

Group
-----

A group of User_ entities can have specific roles in the application.

...

Now that we use references (the User_ format), we can take it one step further and start listing the relations of the entities:

...

User
----

A user entity is used to...

Fields
++++++

:username: The username a user will use to login to his/her account
:email: ...

Relations
+++++++++

:`Group`_: A user can be part of one group.

...

Because we are using the reference notation again, we can already click link to navigate to the different relations.

Now that we have a base document, we can try to generate diagrams of it.

DocUtils

To easily read the information from the RST files, I recommend using DocUtils to generate an XML version of the source file.

rst2xml.py source.rst > ERD.xml

Graphviz DOT files

The Graphviz DOT language is an powerfull format to generate graphs from a text (dot) file.

digraph graphname {
     a -> b -> c;
     b -> d;
 }

The code above will generate the following graph:

A directed graph (source: wikipedia.org)

Putting it all together

Now we can write some code to parse the XML file and generate a dot file out of it.

When we pipe the output to the Graphviz library, we have our final result!

Win!

This image can get quite large, but Graphviz will find a way to position all the entities.

Some notes/thoughts/improvements:

  • Use `Field Lists`_ for fields.
  • Always include “one” or “multiple” in the relation descriptions.
  • Use {timestamp}, {string} notations in the field descriptions to indicate the type.

SOAP-ERROR: Parsing WSDL

If you’re writing or consuming webservices with PHP SOAP, it’s possible you run into the SOAP-ERROR: Parsing WSDL problem once.

The complete error string is:
SOAP-ERROR: Parsing WSDL: Couldn't load from 'http://host/service?wsdl' : <specific error>

If you copy and paste the url (including the ?wsdl parameter) in the browser and you see the WSDL file, the problem lies in the fact that the PHP cannot reach the host.

When you connect to http://host/service, PHP fetches the XML from the WSDL page via fopen(‘http://host/service?wsdl‘) so it can use it to handle the request. In some cases, that request is not routed correct, resulting in the SOAP-ERROR.

Some solutions:

  • Add the hostname in the hostfile of the server (127.0.0.1 hostname).
  • Add the hostname or IP address in the correct VirtualHost (ServerAlias hostname).

You can test the code by adding a file on the server:

<?php
echo htmlentities(file_get_contents('http://host/service?wdsl'));

That way, you know if the server can reach and read the XML file.

Random PHP function explained

file1_Open_Source_PHP_logo_667A few weeks ago, Jamie Bicknell notified me he read my article about the Random PHP function. Because I didn’t publish any code, he wrote a 3 line implementation of how he would fetch the functions for the randomness.

I promised him I would publish the code that weekend, and here we are, a few weeks later…

The script actually parses the http://php.net/quickref.php page, saves the result, and redirects the user.

$cacheFile './functions.tmp';
if (!
is_file($cacheFile)) {
    
$quickref 'http://php.net/quickref.php';
    
$xml DOMDocument::loadHTMLFile($quickref);
    
$links $xml->getElementsByTagName('a');
    
$functions = array();
    foreach (
$links as $link) {
        if (
substr($link->getAttribute('href'), 08) == '/manual/') {
            
$functions[$link->nodeValue] = $link->getAttribute('href');
        }
    }
    
file_put_contents($cacheFileserialize($functions));
} else {
    
$functions unserialize(file_get_contents($cacheFile));
}
$function $functions[array_rand($functions)];
$function 'http://php.net' $function;
header('Location: ' $functiontrue303);
die();

Some improvements:

  • Invalidate the cache file when a new PHP version is announced.
  • Read random lines from the file (instead of loading the complete array in the memory).

Update: Forgot the mention the actual blogpost of Jamie Bicknell…

Stuff to read from Google

Google

Google has some really nice stuff to read, written in an easy to read way, about all sorts of things. It’s a lot about front-end things, but as a back-end developer I’m responsible to build pages & sites with an infrastructure which leverages these things.

The things I found recently: