Lucida Math OpenType

As part of the next incarnation of the Lucida typefaces, I’ve been testing out the OpenType versions of the maths fonts.

It’s fair to say that most people will have seen a Lucida font in one form or another. Lucida has been very popular in the past as one of the very few commercial and unique maths fonts for TeX. It is a super-family of fonts with more font faces than I’m aware of in any other such collection, including serif, sans serif, typewriter, script, blackletter, handwriting, casual, fax (a sturdier serif), and symbols or wingdings. It has been distributed very widely on various computer systems, with Lucida Grande used in Mac OS X itself (the menu font, among other uses), and Java and Windows shipping the standard serif/sans/mono trio for many years.

For more information about the Lucida fonts and their new release, Ulrik Vieth and Mojca Miklavec have just published a TUGboat paper with all the gossip. (Link accessible for TUG members only until it becomes open access in one year.) Further information can also be found on TUG’s own site for the font and an overview of the fonts written by the font designers, Charles Bigelow and Kris Holmes.

Due to my particular interest in Unicode maths, I’m especially excited to see a new OpenType maths font on the scene. I like to think that we’re entering into somewhat of a golden age of maths font design, as OpenType provides a mechanism by which both TeX users and GUI apps such as Microsoft Word can use them; in times past, it was simply rarely worth the expense of creating a new TeX maths font and this is evident by the relative paucity of them.

After the Lucida release later this year, available OpenType maths fonts will include:

• Latin Modern Math (1347)
• XITS Math (2428)
• Cambria Math v1.0 (1592)
• Lucida Math (1947)
• Lucida Math Demibold (877)
• Asana Math (2240)
• Neo Euler (411)

shown with approximate symbol counts for each font (based on symbols defined in unicode-math, not by literal glyph count). The STIX fonts are the reference, here, and contain the largest number of symbols; I believe recent versions of Cambria Math have more than is listed above. But what is impressive to see is that not only does Lucida have a large glyph coverage already, but it will also be provided with a ‘bold’ version of the font as well and, as far as I know, will be the first OpenType maths font to offer this.

To facilitate the use of the bold font, unicode-math can now load multiple maths fonts simultaneously through LaTeX’s \mathversion command (although this new feature hasn’t been widely tested), which is shown in the image above. Note in the image above the integral sign for the bold example isn’t scaling yet; this will be rectified of course for the release version of the fonts.

I’m very pleased to be able to play whatever small role I can in bringing these fonts to new audiences. Many thanks to Charles Bigelow and Kris Holmes for working on the new fonts and everyone involved in the OpenType transition but particularly the seemingly tireless Khaled Hosny. Great work!

Non-textual tabular requirements

I once wrote a procedure for drawing tabulars with square cells; it was one of my earliest experiences with LaTeX programming, actually. When I’d done so, I received a comment ‘why doesn’t LaTeX allow this easily’?

Well, I wondered, why not? My feelings at the time (echoed a little today) were that LaTeX is a tool for writing largely technical documents, and such specific requirements fall outside its regular bounds of ways to typeset tabular material. (Forget, for the minute, that LaTeX’s tables are pretty ugly by default; I’m assuming everybody uses the booktabs package.)

I still agree with the author of booktabs on this matter: ‘It is not going too far to say that if you cannot create a table using the commands in this package, you should redesign it.’ However, I do admit that things like tables with coloured rows and so on do have their uses.

Just recently Karl Berry mentioned he wanted to typeset a grid of images with a large image in the centre. (Not unlike what you see at the top of the page. Just replace the coloured boxes by real images.) This isn’t something that LaTeX does out of the box, and I’m not sure, actually, if any third package can do it either. (I tried but failed with multirow.)

Spurred on by the requisite ConTeXt example that ‘just works’, my own attempt to implement this arrangement turned out to be quite easy but not exactly straightforward. Let me know if there’s a better way.

First of all, the input syntax:

\begin{tighttabular}{@{}c@{}c@{}c@{}c@{}c@{}c@{}}
\1&\1&\1&\1&\1&\1\\
\1&\1&\1&\1&\1&\1\\
\1&\1&  &  &\1&\1\\
\1&\1&\9&  &\1&\1\\
\1&\1&\1&\1&\1&\1\\
\1&\1&\1&\1&\1&\1\\
\end{tighttabular}

Notice that my approach simply puts the material in the lower-left cell in order to fill in the space taken up by the others. This would not work if the cells were of unknown and uneven sizes.

The definition of tighttabular is easy; just define \arraystretch to zero, locally:

\newenvironment{tighttabular}{%
  \def\arraystretch{0}%
  \begin{tabular}%
}{%
  \end{tabular}%
}

How do we place material in that box \9 so it comes out with the correct alignment? Actually, it’s not that bad:

\def\9{%
  \rlap{\smash{\largebox}}%
}

The \rlap first removes the horizontal width, and the \smash removes the vertical height. This is done so that the cell that holds \9 takes up only the same amount of space as the other cells around it. (Otherwise, they would stretch to fit, distorting the size and alignment of the tabular.)

Finally, what is \1 and how do you get the colours to do that?

\def\1{\smallbox}
\def\smallbox{\color{blah!!+}\rule{2cm}{2cm}}
\def\largebox{\scalebox{2}{\smallbox}}

(\scalebox requires the graphicx package.) These boxes use the xcolor package’s very convenient ‘colour series’ feature:

\usepackage{xcolor}
\definecolorseries{blah}{hsb}{step}[hsb]{.5,1,1}{.1,-.05,0}
\resetcolorseries{blah}

And that’s it. Whether you think this whole approach is nice and straightforward or horribly arcane will be somewhat of a personal decision. We’re still awaiting the one tabular package to replace all others in the LaTeX world, although I’m pleased to see recent efforts moving towards providing a complete interface to the different methods supported by the various LaTeX third-party packages in this area.

(With my LaTeX3 hat on: no, as far as I know we’ve not even begun thinking about how this might be dealt with there. I’m not an expert in this area. Although I will say that I mildly dislike both LaTeX’s & and ConTeXt’s \bTR…\eTR syntax; for me, the former is too close to the metal and the latter too verbose.)

The latest version of fontspec now returns proper error messages when a font cannot be found:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
./small.tex:6: 
! fontspec error: "font-not-found"
! 
! The font "Ggeorgia" cannot be found.
! 
! See the fontspec documentation for further information.
! For immediate help type H <return>.
!...............................................  

l.6 \fontspec{Ggeorgia}

? H
|'''''''''''''''''''''''''''''''''''''''''''''''
| A font might not be found for many reasons.
| Check the spelling, where the font is installed etc. etc.
| 
| When in doubt, ask someone for help!
|...............................................

The wording might not be optimal, but having something to explain what’s going on here has been missing for a long time!

The ASCII-art ‘design’ of these error messages comes from work we’ve been doing in expl3 trying to make the console output a little more attractive. It’s not an easy job working around TeX’s idiosyncrasies here, but I’m quite happy with the final result.

The latest issue of the PracTeX Journal (discl.: I used to be a production editor there some years back) contains a set of interesting challenges: typeset and solve a KenKen puzzle. I haven’t seen this puzzle before; it looks fun for those of you who like this sort of thing. I encourage all with the time to attempt at least some of the problem—it looks fun!

In 2008 TPJ issued a similar challenge for Sudoku. At the time I didn’t think anyone would implement a complete Sudoku solution in TeX but I was astonished and happily surprised when two solutions were forthcoming.

It’s good to remember that almost anything can be done in TeX macros if one tries hard enough. In this theme, my favourite example of ‘why not?’ when it comes to theoretically-possible but rather-quite-difficult problem solving in TeX is from the late Michael Downes’ series of ‘Around the Bend’ problems. (Peter Wilson created a wonderfully typeset version available, in a TeX Live distribution at least, via ‘texdoc AroundTheBend’.) See chapters 10 and 11:

Using as few lines of TeX code as possible, set up an Around the Bend post containing a typical exercise so that it can be processed by plain TeX to (a) skip over the exercise text and (b) decode an embedded encoded answer.

It’s a fascinating idea to encode and decode text in TeX itself (of course with LuaTeX around now such ‘problems’ are little more than academic); as far as I know, however, such tools were never exploited for robust text passing to/from other formats. (E.g., inside PDF metadata where data should be encoded correctly.)

And surely everyone reading this has seen David Carlisle’s xii.tex before, right?

Trends in recent years for texhax and the Mac OS X TeX mailing lists. And a comparison with tex.stackexchange.com.

Manuel Pégourié-Gonnard asked whether I’d looked at the number of posts to the texhax mailing list when I posted last night about traffic to comp.text.tex. I had considered it, but I ran out of time, then, to scrape the data from the online archives. See above for the newly-gleaned information.

Interesting enough (and perhaps gratifying) to see that texhax traffic has remained constant over the last eight years; the mailing list itself dates back to 1986 but only recent years were available for my primitive data-analysis skills.

I’ve also included the data for the TeX on Mac OS X mailing list, and there the results are a little more noisy. (Note even three months with greater than 1000 posts per month!) The drop-off in 2007 is interesting, because it correlates with the switch from Gerben Wierda’s ‘gwTeX’ distribution to TeX Live as the basis for MacTeX. Gerben’s work in the 2000s was fantastic, and TeX on the Mac would not have thrived as it did without him—but the move to a centralised TeX Live seems to have been the right choice at that time. (Gerben provided the initial support for TeX Live on the Mac, so his work lives on.)

Since 2007 the data for the Mac OS X TeX mailing list is pretty much constant, tracking slightly above texhax in terms of number of posts per month.

I also mentioned yesterday that I think a fair chunk of the traffic to comp.text.tex may have moved to tex.stackexchange.com. The ‘tex.se’ Q&A site officially launched three months ago after a three month beta period. At that time, the site had on average 12.7 questions per day with 2.4 answers being given per question (numbers from area51).

Three months later, being now, the site receives 24 questions per day, and the answer ratio is slightly lower at 2.1 answers per question.

In total, the site currently contains some 3400 questions and 7200 answers: the equivalent of around 1700 ‘posts’ per month (I don’t have exactly numbers to the day for how long it’s been operating). And this rate seems to be increasing, although two data-points is hardly a trend.

Compared to the traffic for comp.text.tex and the mailing lists I’ve shown above, I don’t think you can argue that tex.se is now playing a very important role for TeX support.

With new LaTeX sites around nowadays such as tex.stackexchange.com and latex-community.org, it seems that the comp.text.tex newsgroup, the mainstay of the TeX support world, is in decline.

Newsgroups in 2011 are an anachronism from the younger days of the internet, and access to them is rather more limited than it used to be. Google provides a web interface to them through Google Groups but in recent years it’s slowly been more and more spam-afflicted.

Is this a indication, perhaps, that TeX use itself is diminishing? I don’t think you can really argue this, as posts on c.t.t. hardly reflect consistent usage numbers over time—if newcomers to TeX and friends find Q&A sites like Stack Exchange I think they’ll be far less likely to want to use Google Groups, which is far more clunky with a poorer signal-to-noise ratio.

But it’s still a shame (or perhaps just a nostalgic twinge) to see a community in decline, even while new ones are coming into being. Despite this, the c.t.t. archives are full of valuable information, and even if new postings diminish the group will live on as a historical record of helpful answers. When in doubt, search!

Funny problem in mathtools vs. francais babel. (Only a coincidence that I’m debugging two babel problems back-to-back.)

Morten Høgholm’s mathtools package has an option to turn the colon into an active character so that writing := produces a colon-equals sign with a correctly centred colon. (Unicode mathematics provides a specific character for this symbol but the input shorthand is still a convenience.)

But babel’s French module also has a feature with an active colon, and although both mathtools and babel can be otherwise used together without problem, these two features cannot both be active simultaneously. This is a documented problem, and a workaround requires changes to babel to allow hooks into its active characters. (Not going to happen any time soon, in other words.)

Despite the documented incompatibility, there was still something breaking when both packages were loaded, and even when the centred-colon feature was never activated. Turns out we were lazy in mathtools. (To be fair, this portion of the code originated in the depths of time on comp.text.tex.)

Active characters are interesting because they can be switched on and off by changing their catcode, and they remember their meaning the whole time. In mathtools, we were taking advantage of this by writing something like

\begingroup
  \catcode`\:=\active
  \gdef:\fancycolon % not the actual defn.
\endgroup

and in the mathtools option processing, all we had to do to activate the centred-colon feature was to execute

\catcode`\:=\active

and our previously saved definition automatically takes into effect.

Unfortunately, this is a rather inconsiderate way to do things, because it doesn’t take into account the fact that other people (in this case babel) might also want to assign the colon an active meaning. If mathtools is loaded after babel, then the colon is overwritten even though the centred-colon feature is never activated!

There are a few ways that both mathtools and babel could behaviour more sensibly here. But when dealing with catcodes, such clashes are inevitable since there are so few ASCII characters and we always want to simplify the way we write our documents.

A user of Peter Wilson’s fonttable package reported an interesting bug in the interaction between it and the Spanish module (if not others) of babel.

Here’s the problem in a nutshell:

\documentclass{article}
\usepackage[spanish]{babel}
\begin{document}
$\lim x_n$
{\font\x=cmr10\x hello}
$\lim x_n$
\end{document}

Can you see the problem? It certainly wasn’t immediately obvious to me.

I probably wouldn’t have had time to work through the problem (after all, it’s clearly babel’s problem, not fonttable’s) but the user that reported the problem had also tracked down the root of it already (thanks!). In Spanish, the maths operator \lim is written as ‘lím’, which requires the use (in 8-bit LaTeX) of an acute accent combined with a dotless ‘i’. And inside the definition for \dotlessi in babel’s Spanish module, you’ll find

\expandafter\expandafter\expandafter
\split@name\expandafter\string\the\textfont\mathgroup\@nil

Note the \split@name command, which takes an NFSS font name such as OT1/cmr10/m/n/10 and split it into its components (resp.: encoding, family, series, shape, size). It’s operating on \the\textfont\mathgroup there, and this is where the problem comes from.

By default in a 10pt LaTeX document, \textfont is defined as OT1/cmr10/m/n/10. In pseudocode, LaTeX originally calls something like this:

\font \OT1/cmr10/m/n/10 = cmr10 at 10pt
\textfont\fam=\OT1/cmr10/m/n/10

(This would not execute without ‘/’ having \catcode ‘letter’.) The problem comes in when someone comes along and writes something like

\font\x=cmr10 at 10pt

Because TeX does not want to load the same font more than once, it optimises its references to font names to point to the last name a specific font was requested with (here ‘font’ means ‘font face + size’).

At this point, the \textfont now points to \x instead of \OT1/cmr10/m/n/10, which causes \split@name to break pretty spectacularly. And this even happens if the font loading happens inside a group:

\font\1=cmr10 at 9pt
{\font\2=cmr10 at 9pt}
\textfont\fam=\1
\showthe\textfont\fam

The above code returns ‘\2’, quite unintuitively. So what can be done about this problem? Let’s assume for now that we can’t change how babel does things (I don’t know the practicality of that in any event).

One way to avoid this problem occurring might be to dynamically check the current ‘values’ of \textfont, \scriptfont, etc., and then not reload the same font under a different name if it’s the same as the font you would like to load. This could be implemented in a wrapper around \font like \newcommand is around \def, and this might be something we look into for expl3.

A less general solution is to simply load the new font like this:

\font\x=cmr10 at 9.9999pt

Because the font is technically at a different size (although the difference is imperceptible), TeX considers it a separate font and will no longer replace its old pointer to \OT1/cmr10/m/n/10 with \x. (I also perform this trick in unicode-math to load the same font with different \fontdimens for assigning to different math groups.)

This is certainly less robust than an explicit check, and will not always be an acceptable solution, but it’s simple to implement in fonttable and serves its purpose there quite well.

In conclusion, due to this fragility in babel (I don’t know if it’s the Spanish module only but it seems likely the problem could occur elsewhere), be careful loading certain fonts with \font unless you are confident it won’t also be used as a maths font. If possible, use the LaTeX font loading interface instead.

You often hear the term “callbacks” bandied about when reading about new functionality offered by LuaTeX. Callbacks offer powerful functionality, but they don’t have pithy explanation that can easily explain why they’re so useful.

Paul Isambert has written a great example of using callbacks on comp.text.tex, his post prompted by Dan Luecking’s plea:

What is a callback? And don’t tell me to read the manual: it has over 100 instances of the word callback and I gave up trying to find out what they are after examining the first 50 or so instances.

Paul’s reply is here; he hints at an upcoming TUGboat article going into more detail (don’t have a TUG subscription? You should: it’s cheap and furthers the development of TeX and friends), but here’s his opening explanation:

[A] callback is a point where you can interrupt TeX’s normal operations, and throw in some (Lua) code of your own. There are many such points, for many different operations: managing the input buffer, reading a file, loading a font, building a paragraph, etc.

His example then shows a short way to do in LuaTeX what requires rather more fragile and tricky code in regular TeX. Good stuff.

John Bowman has just announced version 2.05 of Asymptote, a 3D programmatic drawing package that uses TeX for typesetting its labels. Asymptote is unique in its ability to generate 3D graphics that can be rotated and zoomed within Adobe Reader, allowing far more interactive graphics for scientific documents.

John has written several TUG papers on Asymptote over the last few years; the latest was for the TUG 2010 conference held earlier this year in San Francisco (the video of John’s presentation is available on River Valley TV). I had the great pleasure of meeting John at the conference; amongst other things we discussed some improvements I had in mind for the LaTeX package for including Asymptote graphics within a document.

After the conference I foolishly went over the package and re-wrote much of it in the DocStrip format to better document the internal behaviour of the code. Unfortunately—hence the ‘foolishly’—many bugs were introduced in the process when my new code smoothed over edge cases that it shouldn’t have—but it’s now finally working and included in the main Asymptote distribution. The package source is now more modern and better documented (available in the file doc/asy-latex.dtx in the Asymptote distribution), which should help with maintenance in the long run.

The main reason for the re-write was to add the feature to be able to include graphics defined in external files; before, graphics had to be included inline in the form

\begin{asy}
    asy graphics code
\end{asy}

The new version of asymptote.sty allows you to write

\asyinclude{myfig.asy}

where the asy graphics code is now placed in the file myfig.asy. This can be useful when you wish to re-use an Asymptote graphic between multiple files or if you just like to keep your document source as minimal as possible.

Thanks to John for his patience in testing the packages as bugs were fixed and yet more bugs were discovered. It’s been great to work together.