Fragments

Three days ago I pointed out that the UK government was lying about the influence of the war in Ukraine on UK retail energy prices. Now we have a better idea what that influence might actually be.

The UK government would like you to believe that the recent increases in the rate people pay for energy are due to the war in Ukraine. This is a lie.

They are corrupt, they have done abominable works, there is none that doeth good.

Here’s a simple example of dealing with a naturally circular function definition.

Common Lisp has a predicate called some. Here is what looks like a natural definition of a slightly more limited version of this predicate, which only works on lists, in Racket:

(define (some? predicate . lists)
  ;; Just avoid the spread/nospread problem
  (some*? predicate lists))

(define (some*? predicate lists)
  (cond
    [(null? lists)
     ;; if there are no elements the predicate is not true
     #f]
    [(some? null? lists)
     ;; if any of the lists is empty we've failed
     #f]
    [(apply predicate (map first lists))
     ;; The predicate is true on the first elements
     #t]
    [else
     (some*? predicate (map rest lists))]))

Well, that looks neat, right? Except it is very obviously doomed because some*? falls immediately into an infinite recursion.

Well, the trick to avoid this is to check whether the predicate is null? and handle that case explicitly:

(define (some*? predicate lists)
  (cond
    [(null? lists)
     ;; 
     (error 'some? "need at least one list")]
    [(eq? predicate null?)
     ;; Catch the circularity and defang it
     (match lists
       [(list (? list? l))
        (cond
          [(null? l)
           #f]
          [(null? (first l))
           #t]
          [else
           (some? null? (rest l))])]
       [_ (error 'some? "~S bogus for null?" lists)])]
    [(some? null? lists)
     ;; if any of the lists is empty we've failed
     #f]
    [(apply predicate (map first lists))
     ;; The predicate is true on the first elements
     #t]
    [else
     (some*? predicate (map rest lists))]))

And this now works fine.

Of course this is a rather inefficient version of such a predicate, but it’s nice. Well, I think it is.

Note: a previous version of this had an extremely broken version of some*? which worked, by coincidence, sometimes.

Common Lisp is, I think, a remarkably pleasant language, despite what some people like to say. Here are two small deficiencies, both of which are understandable in terms of the history of CL, and both of which ultimately hurt naïve programmers working in CL.

The default floating-point type is single-float

There are two things that make this true:

• *read-default-float-format* is initially single-float, which means that, unless it is changed, 1.0 reads as 1.0f0, a single float¹;
• The float function will convert to a single float unless it is given a prototype which is not a single float: (float 1) is 1.0f0, while to get a double float you would need (float 1 1.0d0).

In addition things like with-standard-io-syntax bind *read-default-float-format* to single-float, so you have to do a little more work to make doubles the default.

I think there are probably several historical reasons why this default was chosen:

• a long time ago memory was very expensive and single floats take, usually, half the memory of double floats, thus pushing people towards single floats;
• a long time ago, perhaps, on some machines, single float operations were significantly faster than double float operations even before possible float consing was taken into account;
• Lisp hardware companies with significant influence on the standard, notably Symbolics, made hardware which allowed single (32 bit) floats to be immediate objects, while double floats were not, and had simple-minded compilers which were not capable of optimizing double float operations, thus making double float arithmetic extremely slow compared to single float arithmetic, and these companies wanted their machines to seem fast (they never, really, were) for naïve users;
• it was not clear that implementations would choose single-float to mean ‘single precision IEEE 754 float’ and double-float to mean ‘double precision IEEE 754 float’, for instance it’s perfectly legal to have the short-float type mean single precision IEEE 754 and all of the single-float, double-float and long-float types mean double precision IEEE 754;
• it wasn’t even even clear that IEEE 754 would come to dominate how machines implement floating-point: VAXes didn’t, and other machines of interest at the time also did not.

So there are good historical reasons for this. However all implementations I’m aware of now translate short-float to mean single-float, single-float to mean IEEE 754 single precision, double-float to mean IEEE 754 double precision and long-float to be the same as double-float.

So what is the problem with the default float type being single-float in the modern world? The answer is

> (log (/ 1 single-float-epsilon) 10)
7.22472

In other words, single precision IEEE 754 arithmetic has about 7 significant figures of precision. For many purposes, and especially for naïvely-written code that’s at best marginal and at worst less than that. On the other hand

> (log (/ 1 double-float-epsilon) 10)
15.954589770191001D0

which is almost 16 significant figures of precision, more than twice that of single precision.

That’s why the default should have been double precision: it makes naïve code more likely to work, and people who are writing non-naïve code can use single precision if they need it.

The CL-USER package is defined in an implementation-dependent way

From the spec:

The COMMON-LISP-USER package is the current package when a Common Lisp system starts up. This package uses the COMMON-LISP package. The COMMON-LISP-USER package has the nickname CL-USER. The COMMON-LISP-USER package can have additional symbols interned within it; it can use other implementation-defined packages.

(My emphasis.)

What this means is that when you start a CL environment, the current package may have all sorts of implementation-dependent symbols visible in it. You can see why this happened: if you’re implementing Super-Whizz-Bang CL which has all sorts of magic extra features, you want at least some of those features to be immediately available to users, rather than requiring them to pore over boring manuals to find them.

But for users, and especially for naïve users, it’s a terrible choice: naïve users don’t know about packages so they write their programs in CL-USER. And they also don’t really know which symbols available in CL-USER come from CL and are thus standard parts of the language, and which come from one of Super-Whizz-Bang CL’s implementation packages, and are not standard parts of the language. So their programs turn into a mess where the portable parts are not distinct from the non-portable parts. The way the CL-USER package is defined thus makes it harder for to write programs whose non-portable parts are well-isolated, and ultimately hurts the language.

This is a direct conflict between implementors and users: implementors both want their extra features immediately available so their implementation is shinier and want to encourage users to use these extra features in a way which makes it hard to move their programs to other implementations; users, when they think about it, generally don’t want this second thing, at least.

Instead, the language should have defined CL-USER as a package which only used CL, and perhaps have defined another standard package, perhaps IMPL-USER, which was defined the way CL-USER is today.

Can these be fixed?

While both of these problems could be fixed without changing the standard, I don’t think either can realistically be fixed.

For the single-float problem there is nothing to stop implementations simply defining short-float to mean IEEE 754 single precision and all the other types to mean IEEE 754 double precision. But all the existing code which assumes otherwise will then probably break in exciting ways. So this is unlikely to happen I expect.

The CL-USER problem could be fixed if implementations agree to define CL-USER to use only CL as it is allowed to do, and perhaps to define an IMPL-USER package as above. Of course that will make implementations slightly less convenient to use, so the chances of it happening would be small, even if implementors actually talked to each other in any useful way which I suspect they no longer do. Worse than that, this change will break many programs written by naïve users which live in CL-USER, and there are almost certainly lots of those.

A moment of convenience, a lifetime of regret, as the old saying goes.

Putin’s invasion of Ukraine is horrifying. As well as the awfulness of what is happening to the people of Ukraine, Putin’s apparent irrationality is terrifying. What if he is not being irrational?

There are apocryphal reports that Apple M1 systems are not as fast as people have been led to believe for general-purpose programs. That’s unsurprising.

Many people would like to believe that the CV19 pandemic is over. Unfortunately viruses do not listen to what people want to believe: the CV19 pandemic is not over, and there is a significant possibility it may never be over. The way out is not to pretend that it is.

The idiot child god and where he will lead us.

It seems that my article about the existence in the Lisp community of rather noisy people who seem to enjoy complaining rather than fixing things has atracted some interest. Some things in it were unclear, and some other things seem to have been misinterpreted: here are some corrections and clarifications.

First of all some people pointed out, correctly, that LispWorks is expensive if you live in a low-income country. That’s true: I should have been clearer that I believe the phenonenon I am describing is exclusively a rich-world one. I may be incorrect but I have never heard anyone from a non-rich-world country doing this kind of destructuve whining.

It may also have appeared that I am claiming that all Lisp people do this: I’m not. I think the number of people is very small, and that it has always been small. But they are very noisy and even a small number of noisy people can be very destructive.

Some people seem to have interpreted what I wrote as saying that the current situation was fine and that Emacs / SLIME / SLY was in fact the best possible answer. Given that my second sentence was

[Better IDEs] would obviously be desirable.

this is a curious misreading. Just in case I need to make the point any more strongly: I don’t think that Emacs is some kind of be-all and end-all: better IDEs would be very good. But I also don’t think Emacs is this insurmountable barrier that people pretend it is, and I also very definitely think that some small number of people are claiming it is because they want to lose.

I should point out that this claim that it is not an insurmountable barrier comes from some experience: I have taught people Common Lisp, for money, and I’ve done so based on at least three environments:

• LispWorks;
• Something based around Emacs and a CL running under it;
• Genera.

None of those environments presented any significant barrier. I think that LW was probably the most liked but none of them got in the way or put people off.

In summary: I don’t think that the current situation is ideal, and if you read what I wrote as saying that you need to read more carefully. I do think that the current situation is not going to deter anyone seriously interested and is very far from the largest barrier to becoming good at Lisp. I do think that, if you want to do something to make the situation better then you should do it, not hang around on reddit complaining about how awful it is, but that there are a small number of noisy people who do exactly that because, for them, no situation would be ideal because what they want is to avoid being able to get useful work done. Those people, unsurprisingly, often become extremely upset when you confront them with this awkward truth about themselves. They are also extremely destructive influences on any discussion around Lisp. (Equivalents of these noisy people exist in other areas, of course.) That’s one of the reasons I no longer participate in the forums where these people tend to exist.

(Thanks to an ex-colleague for pointing out that I should perhaps post this.)