We don't often add a new tool to our development stack, but when we do it's htmx.
Some years back we started a side project to implement a RESTful api and corresponding web application. Knowing virtually nothing of how the web worked, we remember being shocked and outraged that browsers would not, and would probably never, implement PUT and DELETE. There were some reasonable arguments put forth at the time as to why this should be so, and perhaps they are still reasonable. Anyway, we took the not uncommon path of putting hidden values our html forms for the real method we wanted the server to process. Then one day, around the summer of 2020, we ran into this mild-mannered, unassuming professor in Bozeman who mentioned he was the developer of a JavaScript project called htmx that, what do you know, provided a way for the browser to do PUTs and DELETEs. What are the odds! We immediately investigated htmx and made a branch in our project to start playing around with it. More pressing work soon got in the way, and the effort was abandoned for five years. Now however we are back looking at htmx, which has gone from internet obscurity to Github sensation in the intervening years. And the mild-mannered professor has transformed into a Montana wild man.
Deceptively simple, htmx is actually quite complex. There are many ways to do things. We need to extract some principles or techniques and narrow down the number of features, parameters, and options that we are going to use.
There is this distinction between in-band and out-of-band swaps, which we find not particulary useful. More useful is the distinction between request-driven and response-driven swaps. The attributes hx-swap, hx-target, hx-select, and hx-select-oob are all request-driven. These sit in the HTML in the DOM in the browser along side hx-post and its companions. They are known to the browser before the request is fired off, and instruct htmx how to handle the response.
On the other hand, hx-swap-oob is a response-driven attribute. It has no purpose before the request. This attribute is received in the response body in the HTML, processed by htmx, stripped out of the HTML, and then thrown away never to be seen again. In this sense it functions like a response header. In fact it could easily be implemented as a response header that contained a list of ids.
Then there are actual htmx response headers that convert a request-driven swap to a response-driven swap. HX-Reswap, HX-Retarget and HX-Reselect let the server overide the values of hx-swap, hx-target, and hx-select respectively.
In addition there is a core htmx extension named Response Targets. This supports attributes in the form of hx-target-[code] where [code] is a three digit HTTP response code. This provides another way to implement response-driven swaps.
So response-driven swaps can be implemented either as header-driven overrides or as content-driven HTML, or using the Response Targets extension or some unholy mixture of the three.
The single most obvious advantage of response-driven swaps is that they are dynamic; the server receives a request and can define what gets swapped where back on the client, depending on whatever is appropriate at the moment. The response can contain zero, one or many HTML fragments to be swapped in.
The advantage of relying only on hx-swap-oob for swaps is that all the logic is in one place. We know exactly what is going to happen by looking at the content of the response. There is no need to inspect the response headers or the client side HTML.
Otherwise there is potentially logic in three places: the HTML in the browser, the headers in the response, and the content of the response. All of these directives scattered about can interact and conflict. There are even configuration settings to tweak the default behavior.
It is possible to use hx-select-oob to somewhat mimic the flexibility of hx-swap-oob. We can specifiy multiple ids, and the server can respond with zero of more fragments with those ids. In this case the client must know ahead of time the set of ids that might be swapped, but the server is free to respond with none, some, or all.
Now let's say we want to take it up a notch and add a little JavaScript. On the client side we could use hx-on* to fire off a JavaScript snippet after the request. Or, on the server side, we can use one of the trigger response headers, HX-Trigger, HX-Trigger-After-Settle, or HX-Trigger-After-Swap to fire an event when response is received. Alternatively, we can just include a script element in the response.
In other words, hx-swap-oob can replace using all of:
hx-swap
hx-target
hx-select
hx-select-oob
HX-Reswap
HX-Retarget
HX-Reselect
HX-Trigger*
hx-target-[code]
hx-on*
htmx.config.[various]
No doubt there are use cases for all of these attributes and headers, but if one is in total control of the back end, it seems simpler to just use hx-swap-oob.
Let's make all this a little more concrete. Consider a button that fires off a POST. The basic pattern is a request-driven swap using hx-target:
b←New 'button'
b.hx_post←'/SomeURL'
b.hx_target←'#MyId'
Let the response body be:
<div>
This is the response.
</div>
The response body is swapped into the element with id MyId. There is only one target element, and the entire body of the response is swapped in to this target. The target must be known and specified before the request is sent off.
Multiple elements may be swapped in to different locations by using hx-select-oob:
b←New 'button'
b.hx_post←'/SomeURL'
b.hx_select_oob←'#MyId1,#MyId2,#MyId3'
Let the response body be:
<div id=MyId1>One</div>
<div id=MyId2>Two</div>
<div id=MyId3>Three</div>
These are called out-of-band swaps. The request is again driving the swap. It specifies a list of ids that may be swapped. In order to swap an element, the response must have an element with a corresponding id. The response does not need to contain all or indeed any of the ids specified in hx-select-oob If there is no corresponding id in the response, it's a no-op. This allows for a little bit of response-driven swapping. The server decides what it wants to swap, but within the universe of ids explicitly specified by the request.
A basic in-band swap may be combined with an array of out-of-band swaps, using hx-target and hx-select-oob together.
We can move the explicit list of ids for out-of-band swaps from the client to the server by using hx-swap-oob in the response:
b←New 'button'
b.hx_post←'/SomeURL'
Let the response body be:
<div hx-swap-oob='true' id=MyId1>One</div>
<div hx-swap-oob='true' id=MyId2>Two</div>
<div hx-swap-oob='true' id=MyId3>Three</div>
Now the server has complete control over what gets swapped. The server can send any set of ids in the response. We can combine hx-target and hx-select-oob on the client side, with hx-swap-oob in the response to... confuse ourselves.
So for now, our strategy is to exclusively use hx-swap-oob, use script tags to execute a little JavaScript if necessary, and to configure htmx to swap on any and all response codes.
Back in December on the htmx Discord someone posted the following:
Honest question: whenever I see code snippets that generate HTML using some other language (JavaScript, Python, whenever) I kind of cringe. My only real experience is using Django Template Language, which is basically just writing actual HTML with the benefit of variables and loops. But it very much just feels like writing actual HTML.
What are the benefits of using another language to write HTML? I must be missing something. Chaining together objects and methods named after HTML elements and specifying HTML attributes via class properties just seems unnecessary compared to just writing out HTML using something akin to Django Template Language.
We have been pondering the exact opposite of this question: Whenever we see HTML templating code, we kind of cringe. Our only real experience is using APL, which is basically just writing array-based solutions without the detriment of loops. What are the benefits of using an HTML template language? We must be missing something. Injecting variables and loops and if-statements into HTML just seems unnecessary compared to writing out HTML using APL.
We will note that after a brief investigation into various projects that replace templating with HTML generation directly in the host language, we cringe too!
First it should be noted that embedded in the original question is the admission of the usefulness of using a language other than HTML to write HTML. So the question really is: if you are going to use a language in addition to HTML to write HTML, should it be a template language or a real programming language? Should you put code in the HTML, or HTML in the code?
Why would one want to learn yet another language, just for templating HTML? That is, a language in addition to HTML and your primary programming language? A language sure to be obsolete soon enough? HTML will be around for 100 years. APL will be around for 1000 years. Your template language will be around for 6 months.
One answer is that you might want to create a wide variety of text file formats, not just HTML. At some point learning a third generic text templating language might be worth the effort. If its just HTML, CSV and JSON, however, it's doubtful the benefits exceed the costs. All three of these formats are much better produced with a real programming language rather than templating.
It's not that HTML is a bad language or that we don't like it. We love HTML and we love and CSS. It's just that HTML is not primarily designed for humans to write. In fact, it is for precisely this reason that HTML template languages exist. If HTML was designed for humans to write effectively, we wouldn't need template languages. Other than as a learning experience, why would anyone possibly want to write HTML directly? Writing HTML by hand is a rote, boring, and error-prone task. Editing it is onerous. Consider this example of simple select element, adapted from the MDN documentation:
<select>
<option value="dog">Dog</option>
<option value="cat">Cat</option>
<option value="hamster">Hamster</option>
<option value="parrot">Parrot</option>
<option value="spider">Spider</option>
<option value="goldfish">Goldfish</option>
</select>
The redundancy is overwhelming.
Of course there are editors and IDEs that help with all the duplication. But that is itself a testament to the fact that HTML should not be written by hand. Editors that automatically add ending tags, or that allow multi-line editing and multiple cursors are fixing problems that generally shouldn't exist. They are making up for deficiencies in language design, bad coding styles, or writing code in languages that should not be written by hand.
Let's do this in a proper programming language. Here APL, but no doubt many other interpreted languages are just as good. First, define the values and the descriptions:
v←'dog' 'cat' 'hampster' 'parrot' 'spider' 'goldfish'
d←'Dog' 'Cat' 'Hampster' 'Parrot' 'Spider' 'Goldfish'
Then define the elements and their content:
s←New 'select' (New¨d{'option' ⍺ ⍵}¨{'value' ⍵}¨v)
Ok, that's a bit ridiculous-- way too complicated. We would not write it that way in production, but would break it up like so:
o←New¨{'option' ⍵}¨d
o.value←v
s←New 'select' o
And create the HTML using the Render function:
Render s
<select>
<option value="dog">Dog</option>
<option value="cat">Cat</option>
<option value="hampster">Hampster</option>
<option value="parrot">Parrot</option>
<option value="spider">Spider</option>
<option value="goldfish">Goldfish</option>
</select>
There is virtually no duplication or boilerplate in the code. Let's break this down. The New function creates any arbitrary element:
Render New 'option'
<option></option>
New accepts an optional second argument, which may be simple text:
Render New 'option' 'Dog'
<option>Dog</option>
Attributes may be specified as an optional third argument to New as we do in our one-liner above, or specified after an element is created by assignment:
a←New 'option' 'Dog'
a.(value id)←'dog' 'Cleo'
Render a
<option id="Cleo" value="dog">Dog</option>
We can create an array of option elements in one go using an annonymous function and the each (¨) operator:
o←New¨{'option' ⍵}¨d
The variable o is a six-item array of option elements. We can inspect the content:
o.Content
┌─────┬─────┬──────────┬────────┬────────┬──────────┐
│┌───┐│┌───┐│┌────────┐│┌──────┐│┌──────┐│┌────────┐│
││Dog│││Cat│││Hampster│││Parrot│││Spider│││Goldfish││
│└───┘│└───┘│└────────┘│└──────┘│└──────┘│└────────┘│
└─────┴─────┴──────────┴────────┴────────┴──────────┘
and render the array of elements:
Render o
<option>Dog</option>
<option>Cat</option>
<option>Hampster</option>
<option>Parrot</option>
<option>Spider</option>
<option>Goldfish</option>
Assuming a variable p containing a default selection we can specify the selected item:
p←'parrot'
i←v⍳⊂p
(i⊃o).selected←'selected'
and render:
Render o
<option>Dog</option>
<option>Cat</option>
<option>Hampster</option>
<option selected="selected">Parrot</option>
<option>Spider</option>
<option>Goldfish</option>
The value attribute may be specified for all of the elements:
o.value←v
Render o
<option value="dog">Dog</option>
<option value="cat">Cat</option>
<option value="hampster">Hampster</option>
<option selected="selected" value="parrot">Parrot</option>
<option value="spider">Spider</option>
<option value="goldfish">Goldfish</option>
Finally we can create the select element with the options as its content:
s←New 'select' o
And render again:
Render s
<select>
<option value="dog">Dog</option>
<option value="cat">Cat</option>
<option value="hampster">Hampster</option>
<option selected="selected" value="parrot">Parrot</option>
<option value="spider">Spider</option>
<option value="goldfish">Goldfish</option>
</select>
A templating language can help remove much of the duplication when iterating, for example, through the content of list, select, and table elements, but removing duplication is not the most important aspect of writing HTML using a real programming language. HTML, and any templating language based on it, is not executable. It's a text representation suitable for sending over the wire, but you can't do anything with it in your programming environment.
A line of HTML by itself does not do anything. In fact HTML does not even have lines; it is broken into lines above just for clarity. The entire file must be processed before it can be made sense of. The APL code on the other hand is executable. Each line does something. We can run the code, stopping after each line to see what it does. We can write:
p←New 'p' 'Hello world!'
And see that p is:
p
#.Abacus.Main.[Namespace]
...a plain APL namespace. And then set a variable in p:
p.class←'some-class'
And apply a function to p:
Render p
<p class="some-class">Hello world!</p>
This is one reason the browser parses the HTML into a tree of elements; JavaScript can then be used to manipulate the DOM. There is no reason the same power, or much more power really with an array language like APL, should not be available in the programming language of your choice.
We have looked at only two functions so far, New, which creates a new element, and Render which renders an element or a vector of elements. Another useful function is Elements which takes an element and returns a vector of the element itself and all of its descendants, in depth-first order. To show what we can do with Elements, let's introduce one more function, NewTable, a simple cover of New, to create tables:
t←NewTable (⍕¨4 3 ⍴⍳10) ('Col1' 'Col2' 'Col3')
Render t
<table>
<thead>
<tr>
<th>Col1</th>
<th>Col2</th>
<th>Col3</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
<td>2</td>
</tr>
...
<tr>
<td>9</td>
<td>10</td>
<td>11</td>
</tr>
</tbody>
</table>
Working with tables highlights the benefits of using a real programming language to create and manipulate HTML. The elements of table t:
e←Elements t
How many are there:
≢e
23
What are all the tags? What are the unique tags?
e.Tag
table thead tr th th th tbody tr td td td tr td td td tr td td td tr td td td
∪e.Tag
table thead tr th tbody td
Flag the row elements; how many are there?
e.Tag∊⊂'tr'
0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
+/e.Tag∊⊂'tr'
5
Extract the rows just in the body:
b←⊃e/⍨e.Tag∊⊂'tbody'
be←Elements b
r←be/⍨be.Tag∊⊂'tr'
How many are there? Assign a class to them:
nr←≢r
nr
4
r.class←⊂'body-row'
And render:
Render t
<table>
<thead>
<tr>
<th>Col1</th>
<th>Col2</th>
<th>Col3</th>
</tr>
</thead>
<tbody>
<tr class="body-row">
<td>0</td>
<td>1</td>
<td>2</td>
</tr>
...
<tr class="body-row">
<td>9</td>
<td>10</td>
<td>11</td>
</tr>
</tbody>
</table>
Extract the cell elements from the table; how many columns are in the table?
c←be/⍨be.Tag∊⊂'td'
nc←nr÷⍨≢c
nc
3
Make a matrix out of the cells:
m←c⍴⍨nr,nc
Inspect the content:
m.Content
0 1 2
3 4 5
6 7 8
9 10 11
Pick out the bottom right cell and assign it a class:
(⊃⌽,m).class←'current-cell'
And render
Render t
<table>
<thead>
<tr>
<th>Col1</th>
<th>Col2</th>
<th>Col3</th>
</tr>
</thead>
<tbody>
<tr class="body-row">
<td>0</td>
<td>1</td>
<td>2</td>
</tr>
...
<tr class="body-row">
<td>9</td>
<td>10</td>
<td class="current-cell">11</td>
</tr>
</tbody>
</table>
Once we have a DOM in our programming language, we can write functions that create elements and use functions to manipulate an entire document or document fragment. We can create a document or fragment from the outside in or from the inside out. Most importantly we can apply hard-earned and long-accumulated knowledge of code organization, data manipulation, algorithms, and idioms in our chosen language to do it.
We recently had to revamp find and replace functionality, similar to Excel, in a grid. The original implementation predated interval index so it was time for a fresh look. Consider a Boolean matrix representing the cells in a grid where a string or substring is found:
b←.9<?10 7⍴0
b
0 0 0 0 0 0 0
1 0 0 0 0 0 0
0 0 0 0 1 0 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 1 0 0 0 0
1 0 0 0 0 0 0
0 0 1 1 0 0 0
Given any location in the grid, we want to be able to find the location of the next 1. The meaning of "next" will depend on whether searching by row or by column.
The location of ths 1's is given, not coincidentally, using the same glyph as interval index, but in its monadic form where:
i←⍸b
i
┌───┬───┬───┬───┬───┬───┬───┐
│1 0│2 4│4 1│7 2│8 0│9 2│9 3│
└───┴───┴───┴───┴───┴───┴───┘
Given a current location of say, row 7 and column 0:
k←7 0
then the next 1, when searching by rows, is located at:
(1+i⍸⊂k)⊃i
7 2
When searching by column we can transpose the Boolean matrix and find the 1's:
j←⍸⍉b
j
┌───┬───┬───┬───┬───┬───┬───┐
│0 1│0 8│1 4│2 7│2 9│3 9│4 2│
└───┴───┴───┴───┴───┴───┴───┘
then the next 1 is given by:
⌽(1+j⍸⊂⌽k)⊃j
8 0
Note that j may be computed directly from i by rotating and sorting:
{⍵[⍋⍵]}⌽¨i
┌───┬───┬───┬───┬───┬───┬───┐
│0 1│0 8│1 4│2 7│2 9│3 9│4 2│
└───┴───┴───┴───┴───┴───┴───┘
Note further that interval index is happy to trade depth for rank:
m←↑i
m
1 0
2 4
4 1
7 2
8 0
9 2
9 3
m⍸k
2
m[1+m⍸k;]
7 2
Which is quite nice. Thank you Roger!
For finding the previous 1, simply elide the 1+, but an adjustment will need to be made if the current location contains a 1.