Programming Guidelines

Please read through these guidelines when developing for BlattWerkzeug.

Code formatting

Code-formatting for the client folder is automated and enforced by prettier. You may either run make format-code in the client folder manually or setup your IDE or editor of choice to run prettier automatically: Official guidelines for editor integration.

Typical development setup

It is recommended to use four terminal windows when developing, that each show the output from one of the following Makefile targets:

  • Targets in the client folder:
    • Run make client-compile-dev in the client folder. This will starts a filesystem watcher that rebuilds the client incrementally on any change, which drastically reduces subsequent compile times.
    • Run make client-test-watch to continously run the client testcases in the background.
  • Targets in the server folder:
    • Run make reset-live-data dev-make-guest-admin run-dev if you have pulled any seed data changes that need to be reflected. The dev-make-guest-admin target is optional, but very convenient during development.
    • Run make test-watch to continously run the server testcases in the background. This requires a running PostgreSQL database server.
_images/dev-four-console-setup.png

By using this setup you ensure that all tests are run continously, no matter whether you are working on the server or on the client.

Tests, the CI-server and code coverage

Every git push will trigger a build on a CI-server that uses the Docker-images that are part of the repository. But you can (and should!) run the testcases locally, preferably all the time, but at least before a commit.

Client side testing

Calling make test in the client folder will run the tests once against a headless instance of Google Chrome and Firefox. Technically this works by passing the --headless flag when starting the browser, which suppresses any windows that would normally be shown.

  • make test-watch will run the tests continously after every change to the clients code.
  • The environment variable TEST_BROWSERS controls which browsers will run the test, multiple browsers may be specified using a , and spaces are not allowed. The following values should be valid:
    • Firefox and Chrome for the non-headless variants that open dedicated browser windows.
    • FirefoxHeadless and ChromeHeadless that run in the background without any visible window.

After running tests the folder coverage will contain a navigateable code coverage report:

_images/dev-coverage-client.png

Server side testing

Tests for the server are run in the same fashion: Call make test in the server folder to run them once, make test-watch run them continously. And again the folder coverage will contain a code coverage report:

_images/dev-coverage-server.png

Description files and JSON schema

The server and the client need to agree on “over the wire” JSON-objects in order to function properly. As the server is using Ruby on Rails and the client is written in Typescript, this gap is bridged using JSON-schema. All of the relevant schemas live in the schema/json folder.

Updating schemas

The schemas are generated using an automatic conversion process based on Typescript interface definitions. By convention every file that ends on description.ts is a file that is meant to be used as an input for schema generation. If such a file is edited, the affected schemas need to be regenerated by running make all in the schema/json folder.

Rules for description.ts files

  • They define interfaces that are used when communicating between server and client.
  • They must not import complex libraries such as rx.js, Angular, …
  • Because other parts of the projects may import such complex libraries, the description.ts files may only import other description.ts files.

Using JSON schema to validate client requests

When clients want to make a request to the server, they usually need to construct a object that satisfied the relevant description-interface. For this part of the request, Typescript ensures just fine that the data that is send over will actually be understood and valid.

On the server a controller must use JsonSchemaHelper#ensure_request. As with any helper, the JsonSchemaHelper must be referenced via include. If a specific controller requires a request to be made with a TableDescription document, the source code would read as follows:

table_description = ensure_request("TableDescription", request.body.read)

The ensure_request method will take any string, parse it and validate it against the required schema and then return the corresponding hash with the data. If the given string is syntactically valid JSON but does not conform to the schema, an exception is raised and the request is aborted with a 401 Bad Request response.

This procedure ensures that at least the structure of the passed in document is sound. There should be no need to re-validate the structure on the server or to program defensively in order to mitigate missing keys. Even if somebody sends request with arbitrary garbage, these should be filtered out by ensure_request.

Using JSON schema to validate server responses

Because the client is only expected to work with a conforming server, there is no response-validation infrastructure in place during the “nomal” execution of the client. Instead the results of HTTP-requests are simply casted to the expected interface.

The server side request tests ensure that the server responds with conforming documents. A custom rspec validator validate_against should be used with every testcase that expects a specific response.

Using JSON schema to validate models

The backing PostgreSQL database uses a few tables that store jsonb-blobs. After all, PostgreSQL is the best NoSQL database that is available 😉 The jsonb-columns are used for complex and self contained data structures such as the syntaxtrees for a code resource. To ensure that the database does not degenerate overall, the custom json_schema validator for Active Model ensures the validity of all stored blobs.

Loading and storing seed data

BlattWerkzeug comes with a complex set of required objects to work properly. This includes grammars, block languages, example projects, … The “normal” Rails way of providing those objects via db/seeds.rb does not work for these structures at all: They are simply to complex to be meaningfully edited by hand.

The Makefile therefore exposes the store-live-data target which stores the current state of the programing languages and projects in the seed folder. This allows programmers to edit grammars, block languages and projects using the web-IDE and to persist those changes in the git repository.

Important

The YAML-files in the seed-folder are very prone to merge conflicts. Please make sure to only ever commit as small changes as possible. It is good practive to routinely use make reset-live-data run-dev when starting the server to ensure that your database-state is always up do date. If you run store-live-data from an old database state you may override newer changes that are part of the repository already.

Interactive Debugging

The preferred way to figure out the reason for undesired behaviour is by writing testcases: This ensures that the problem does not resurface later as a regression. But if you don’t understand at all why something is going wrong, an interactive debugger is of course helpful.

Client Application

At least the normal development tools of Firefox and Chrome are capable of debugging the Angular application. Depending on your workflow, the debugger statement (Documentation at MDN) may be helpful to set breakpoints directyl from your editor of choice.

Client Tests

You may run the testcases interactively by surfing to http://localhost:9876/debug.html while make test-watch is currently running. This will take you to a page that runs all activated testcases directly in a browser.