Jonty Newman

Creating My Own Web Technologies

In 2013, I became inspired to create my own web technology stack. It had occurred to me that I was consuming many frameworks and libraries with only a surface-level understanding for how they operate. However, rather than study an existing set, I decided that it would be far more engaging to build custom solutions and discover the challenges involved for myself. I would then put these implementations to the test by using them to form my own personal website.

A Framework Sceptic

Early on in my career as a software developer, I began to recognise frameworks as a somewhat necessary evil. Although they can facilitate rapid development, they can also impose additional constraints by mandating how business rules are modeled. For example, a business object may be required to implement a certain interface or be of a certain subclass in order to be recognised and integrated. This results in tight coupling between the business logic and the chosen framework. This was a point of frustration for me, as it was my desire to model the application in a manner that made no reference to dependencies, hence truly separating concerns.

I was therefore glad to discover the concept of clean architecture. It was gratifying to know that I was not alone in my concerns regarding frameworks, and that there was at least one alternative approach that mitigated these costs. It was also around this time that I discovered fellow proponents of why libraries are better than frameworks, who are motivated by the desire to decouple. These principles guided me in the development of the components for my website, so as to reduce the cost of changing layers of the technology stack at a later date.

Ambitions

I had a number of ambitions for the project to determine its success. These ambitions were as follows.

One of the critical decisions in ensuring success was the choice of programming language. At the time of starting the project, I had had extensive exposure to PHP and Ruby on Rails. A mentor during my internship had also suggested the idea of implementing such a project in C.

I decided to research the most prevalent web technologies at the time, and according to W3Techs, this was (and still is) PHP by a significant margin. On this basis, I selected PHP as the programming language that I would use, with the aspiration being that its abundant use on the Internet would maximise the number of potential environments that the project could operate in.

ShrooPHP

I began by working on the outer circle of the clean architecture. The component in this layer would be responsible for interfacing the application with the World Wide Web. I would dub this functionality ShrooPHP (originally standing for simple, highly reconfigurable, object-oriented PHP). In order to ensure a simple, clean and convenient solution, I sought a library that would facilitate a single file of configuration as code. This would therefore impose no directory structure, allowing for greater flexibility.

Configuration as Code

The following is an example of a ShrooPHP application that responds to GET /greetings/* requests with the text Hello, '<subject>'!, where <subject> is extracted from the suffix of the path. The application will also respond to GET /coffee with a 418 client error.

<?php

use ShrooPHP\Core\Request;
use ShrooPHP\Framework\Application;
use ShrooPHP\Framework\Request\Responses\Response;

require 'vendor/autoload.php';

$app = new Application();

$app->method('GET', function (Application $app) {

  $app->path('/greetings/{subject}', function (Application $app) {
      
    $app->push(function (Request $request) {

      $subject = $request->args()->get('subject');

      return Response::string("Hello, '{$subject}'!", 'text/plain');

    });
  });

  $app->path('/coffee', function (Application $app) {

    $response = Response::string("I'm a little teapot.", 'text/plain');
    
    $response->setCode(Response::HTTP_I_AM_A_TEAPOT);

    $app->push($response);

  });
});

$app->run();

The structure of the application is implied by the nesting of callbacks, and is as follows.

The "push" method adds the given request handler to the application's internal list. A request handler can be either a callable or an object implementing the \ShrooPHP\Core\Request\Handler interface. In either case, it is able to return an object implementing the \ShrooPHP\Core\Request\Response interface.

The invocations of the "method" and "path" methods are examples of request handlers being wrapped in multiple conditions. The application achieves this by making use of an internal stack.

When $app->method('GET', …) is invoked, a pointer to the current list of handlers is pushed onto the stack. The current list is then replaced with a new instance before the given callback is invoked. Therefore, $app->path('/greetings/{subject}', …) is also called, and a pointer to the now current list is once again pushed to the stack before the list is replaced with a new instance. The nested callback is then invoked, resulting in this new list having one handler pushed to it by the call to $app->push(…).

When the nested callback completes, the current handler list is buffered while it is replaced with the one at the top of the stack via a "pop" operation. The buffered list is wrapped in a handler that is capable of matching the /greetings/{subject} path and extracting the <subject> from it to add as an "argument" of the request. This wrapping handler is then added to what is now the current list.

The outer callback continues to be processed, and $app->path('/coffee', …) is evaluated in a similar manner to $app->path('/greetings/{subject}', …). When the outer callback completes, the current list is once again replaced by the result of a pop operation on the stack while it is buffered. At this point, the stack is now empty once again. This latest buffered list is wrapped in a handler that is capable of detecting a GET method on the request. Finally, this wrapping handler is added to the application's internal list.

By having an internal stack that reflects the call stack in this manner, the nesting of callbacks can declare the structure of the application logic, hence facilitating configuration as code.

Separation of Concerns

The functionality of ShrooPHP is split across multiple packages.

A separate core allows other libraries to integrate with ShrooPHP, and a separate pattern matching library allows for it to be leveraged elsewhere, independently of the framework.

ShrooPHP\RESTful

In order to assist with creating RESTful applications, I decided to create a library based on ShrooPHP. It provides interfaces for resources and their collections, as well as the necessary request handler and common implementations.

For example, the FileSystem collection can be used to perform CRUD operations on File resources within a directory on the server. There is also a HashTable collection for runtime resources. Although the eventual content management system would not strictly be a RESTful application, this library would be leveraged in order to serve certain files.

ShrooPHP\PSR

ShrooPHP is highly useful, but it lacks interoperability with other projects on its own. To achieve this, a PSR-compatibility layer was implemented as an additional library. It allows ShrooPHP request handlers to be wrapped in an implementation that is compliant with PSR-15. This would prove to be highly useful in the development of futher layers in the clean architecture, as it would allow ShrooPHP to be leveraged without exposure, hence separating concerns.

JontyNewman\Oku

Oku forms the inner-most circle of the clean architecture, and is therefore the most abstract. It achieves this by primarily establishing principles through standardised interfaces. The most critical of these is the concept of a repository.

The following is an abstract example of an Oku-compliant repository.

<?php

use JontyNewman\Oku\ContextInterface;
use JontyNewman\Oku\ResponseBuilderInterface;
use Psr\Http\Message\ServerRequestInterface;

interface ExampleCallableInterface {

  public function __invoke(
    ResponseBuilderInterface $builder,
    ContextInterface $context
  ): void;
}

abstract class ExampleRepository implements ArrayAccess {

  public abstract function offsetGet($key): ExampleCallableInterface;

  public function offsetSet($key, $value): void {

    if (!($value instanceof ServerRequestInterface)) {
      throw new InvalidArgumentException();
    }

    $this->set("{$key}", $value);
  }

  protected abstract function set(
    string $path,
    ServerRequestInterface $request
  ): void;
}

An Oku-compliant repository is an object implementing ArrayAccess. It is expected to receive keys as the path of a given request. When an offset is set, the repository should expect to recieve a PSR-compliant server request as the value. However, when an offset is retrieved, the repository is expected to return a callable value. This callable is expected to receive a JontyNewman\Oku\ResponseBuilderInterface as the first argument and a JontyNewman\Oku\ContextInterface as the second.

These repositories therefore encapsulate the concept of HTTP CRUD operations in a manner that is agnostic to the method of persistence. It is even agnostic to the method of serving the response, as the callable is to accept any implementation of the ResponseBuilderInterface. This facilitates a great deal of reusability for the Oku library.

A number of concrete implementations are available.

The constructor of the request handler optionally accepts a callable that represents editing capabilities. When specified, all content is served using this callback, and allows content to be updated via the HTTP PUT method (or a form POST with a value that overrides the method). In fact, it is from this concept that Oku gets its name, as "oku" in japanese roughly translates to "put".

To bypass the editor, a specific query parameter with a non-empty value can be sent with the request (by default, this is "_inset"). This can allow the content to be rendered alongside the editor if necessary.

JontyNewman\CMS

The content management system is a specific configuration of Oku, and is hence the second-most inner circle of the clean architecture. It combines aforementioned I/O, upload, and data repositories as an aggregate. It is encapsulated in a single class.

For assets (i.e. immutable resources), there exists an asset method that allows them to be specified.

$cms->asset('/style.css', '/filepath/to/asset.css', 'text/css');

This leverages a ShrooPHP RESTful HashTable to set the asset as a runtime resource. If the request matches a specified asset path when $cms->serve() is invoked, that resource is served from the collection. Otherwise, each of the three repositories are queried in turn by the aggregate.

JontyNewman\Website

The website is a specific configuration of the content management system.

Most interestingly, the I/O repository is configured to firstly process input as Twig, then as CommonMark-conformant Markdown. This therefore combines the flexibility of Twig templates with the usability of Markdown in a unique manner.

The content management system allows a specific path to be set as the default, which will be served with a 404 status code if an equivalent resource cannot be found in the aggregate repository. It is hence possible to use this path to persist the block layout and common macros, as in the following example.

<!DOCTYPE html>
<html lang="en-GB">
<head>

{% block head %}

<title>{% block title %}Jonty Newman{% endblock %}</title>
<meta name="viewport" content="width=device-width">
<link rel="shortcut icon" href="/favicon.ico?_inset=1" type="image/x-icon">
<base target="_parent">

{% endblock %}

</head>
<body>

{% block body %}

<header>

{% block header %}

# [Jonty Newman](/)

{% endblock %}

</header>

<main>

{% block main %}

## 404 – Not Found

The requested resource is currently unavailable.

{% endblock %}

</main>

<footer>

{% block footer %}

This web page by [Jonty Newman](/) is licensed under
[CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0).

{% endblock %}

</footer>

{% endblock %}

</body>
</html>

{%- macro day(day) -%}
  <time datetime="{{ day|date('Y-m-d') }}">
    {{- day|date('j<\s\u\p>S</\s\u\p> F, Y')|raw -}}
  </time>
{%- endmacro -%}

{%- macro month(month) -%}
  <time datetime="{{ month|date('Y-m') }}">
    {{- month|date('m/Y') -}}
  </time>
{%- endmacro -%}

The backend of the website provides a Twig function that converts a given request path to the filepath of the equivalent template. This allows the default template to be extended by other pages such as the home page.

{% extends filepath('/default') %}

{% import filepath('/default') as macros %}

{% block main %}

## Hello, world!

Welcome to my website.

{% endblock %}

The combination of Markdown with Twig inheritance results in highly legible templates without the need to duplicate the layout.

Retrospective

It is a pleasure to be able to write articles such as this using Markdown and Twig. And since all pages, data and uploads are persisted using the filesystem, I am able to track changes to my website using source control with incredible ease. It is also a comfort to know that I am able to replace any layer of the architecture if necessary. For example, I may want to replace the outer layer with something more robust such as Lumen.

In hindsight, having a single system that is responsible for both mutating and serving content causes additional complexity. As persistence is fundamentally achieved using the filesystem, there is no reason why this could not instead be two separate systems. This would not only simplify the branches of the codebase, but would also enhance the security of the application.

ShrooPHP could perhaps benefit from being influenced by Oku in terms of agnosticism. Instead of the many interfaces it defines, it could instead rely on callable values and PSR compliance. I can envision myself perhaps someday creating a "JontyNewman\Microframework" package to this end.

Ultimately, the agnostic nature of Oku means that any system can be built and leveraged, so long as it is capable of serving uploads and data, along with processing templates as Twig and Markdown. In other words, the content of the website has very little dependence on the technologies being used to serve it. This is its greatest strength, and ensures a simple, clean and convenient solution.

Conclusion

The experience of building an entire technology stack for my own website has been highly beneficial and gratifying. I now have a manner in which to write webpages which is ideal for me, as well as a better understanding for clean architecture and how trivial frameworks become within it. I can recommend that developers try this experience for themselves; you may very well build something you're proud of.

www.000webhost.com