Web Programming

WebPro

The World Wide Web is over 20 years old technology and can now be called a fundamental platform for software development. In the early days it was not clear that having a web interface would be the appropriate thing for enterprises. Today, it is the essential form. It is so important that we can consider the web as the default and standard interface to enterprise application, be they for desktop computers or mobile devices. My 10th lecture in my Design and Implementation of software is about Web Programming.

When the web started to gain popularity, mostly with static sites manually edited, most enterprise programs were client-server programs. These programmes were installed on users (employees) machines. Microsoft Windows was in the 90s the most dominant operating system so enterprise software was usually Windows clients that accessed some servers that had access to the database.

Most enterprise developers ignored the web as user interface since it was very primitive and clearly a step back. Each interaction, for example pressing a button on the web page, resulted in sending some request to the web server, generating a new page and sending it back. Compared with native Windows program, responsiveness was horrible for the web. Add to this the general scepticism on security issues.

However, the benefits of the web – its simplicity and universal availability outweigh the drawbacks and slowly people started to use the web as the preferred platform.

First programs were CGI programs. CGI stands for Common Gateway Interface. This allowed web servers to execute programs and use the output of the program as a response to the web browser. CGI could be written in any programming language but Perl became popular, not only because it is flexible and powerful, mostly because it is interpreted and does string manipulations very well.

The first programs that I wrote for the web were C++ programs. In the first versions we simply hardcoded the layout and the HTML within the program. Later we switched to Java and JavaServer Pages. Todays programs are much more complicated. We pages are broken into many components and interactivity has been added with JavaScript. As an example, at Betware we use content management system (CMS) with JavaScript widgets that call REST services in the backend.

All these technologies use very simple and fundamental design pattern that first appeared in the 70s, Model View Controller. The idea is to break user interface components into three distinct roles. The Controller is the entry of the request, and handles the parameters that come with the request. The controller uses the model for the “state” or domain logic, be it calculations or getting some data. The model will deliver some data that needs to be rendered. It is the responsibility of the view to render the user interface.

In the lecture, the Model View Controller (MVC) design pattern is covered and some ways to create the controller and the view.

The lecture can be found here: L10 Web Programming

 

 

Behavioural Programming

BehaviorProb

One of the interesting types of programming is tracking behaviour. Most enterprise design patterns are about structure. For example, the Data Source Layer patterns introduced in lecture L08 are structural patterns. They deal with moving data to and from the database. They try to solve the impedance mismatch problem. However, they do not have any logic as to when and under what circumstances data should be loaded to from the database and when written. Clearly accessing the database is slow, and doing so needlessly hurts performance. This is where behavioral programming comes in.

Consider the problem of maintaining multiple objects in memory and working with them. Having updated them we need to persist them back. One way to do this is to load the object from the database, change them, then persist them again. Clearly that is easy to do and avoids any possible errors, for example forgetting to save the object back. Most request-response sites would do it like this. But what if you want to keep the objects in memory and write them to the database when the session is over? One way is to just flush everything out to the database but that does not take into account that many of the objects might not have changed. Then how would we keep track of objects that are changed and those that are not. The solution is to use Unit of Work design pattern.

The Unit of Work (UoW) pattern defines a data structure that keeps track of objects and their state. When an object is loaded, it is registered as clean within the UoW. Any update to the object and the status is changed to dirty. When time comes to save objects, the UofW knows which objects are changed and can save them to the database.

Another problem is due to loading of multiple objects. Let’s say you need to load objects and it may happen that the same object is loaded twice. This can cause multiple of problems, not only space and time complexity but also the risk of concurrency problems as two or more objects might be updated in memory with different updates. To solve this we can use Identity Map (IM), a data structure that holds all objects loaded. Whenever an object is needed we can consult the IM, get it there, but if it is not there, get it from the database and put it in the IM. Clearly combining the UoW and IM is a good choice. As a bonus, the IM works as a cache as objects that are needed multiple of times, are already in memory.

The third problem is with loading objects that have references to other objects. So if you load an orders object (to take the classical example) should you load the customer object also since the order object has a reference to the customer. One way to solve this is not to have object references to other objects in general but use ids and keep each object separate. However, we find ourselves with a solution which has a rich object graph and we need to solve this loading problem. This is where the design pattern Lazy Load (LL) comes in handy. With LL we place markers in the object indicating that the reference is not loaded, but when a get method is called, we will load the referred object and change the marker to indicate that the object is loaded.

In lecture L09 Behavioural Programming we look at these behavioural problem. How to solve issues like writing only changed record back into the database, loading only the needed data once, and how to load the data model partially. We introduce the three design patterns mentioned: Unit of Work, Identity Map and Lazy Load.

L09 The Behavioral Problem

 

Data Source Layer

DataSourceLayer

One of the necessary evils in enterprise application architecture is the relational database. Databases go back long time and are needed to persist data of applications. Relational databases are well-known and widely accepted as the way to persist data. These type of databases date back to the 1960s and are today well established. SQL is common language that most programmers are familiar with. When building enterprise applications the programmer who thinks in objects and their relationships must map the object model to this relational data structure. This is the topic of my latest lecture in my Design and implementation of software course, Lecture L08 Data Source Layer .

There are many well known challenges in working with databases. They tend to dominate design of applications and can become the main part of a system even though they should only persist data and most of the work is in the domain. Another problem with relational database in particular is that the object world and the relational world of tables and rows are quite different. This is what we call the impedance mismatch.

The most common way to organise programs to access the database is to create a new layer in the system, Data Source Layer, also called Persistence Layer. The idea is then to move the database code into a separate layer and away from the application domain. Very often the domain and the data model are similar so this might sound like a duplication of effort.

The simplest and most intuitive way is to create a gateway to each table (or set of tables). This is called Table Data Gateway which is similar to DAO pattern made famous in the Microsoft world. We can also have a gateway to each row, Row Data Gateway. This is useful when working with one row in a request for example, but it becomes a overkill if we need to work with many rows to have a class for each row. In cases where the domain model and the data model  are independent, we can use a Data Mapper which is based on the mapper base pattern. It becomes the responsibility of the mapper to talk to both the domain model and the database. Finally, a simple way to work with a database is to have Active Record. The idea is that an Active Records contains all the fields in a database table and knows how to load itself and save itself.

See lecture:

Lecture L08 Data Source Layer 

Flickr cover image: orangejon