## Sunday, December 16, 2012

### A Flat Dark Theme for Unity/Gnome 3

I care a fair amount about the style and visual nature of my environment, not only from an appreciation of nice things, but also from a utilitarian standpoint. I use Unity and up until today, have been using a theme called Zukitwo, in particular the "brave" version of that theme. I used this as it was reasonably dark and stylish. Today I switched to a darker theme (the windows contents are actually dark), and one that I think I like a bit better, Boje. If you actually kind of like the minimalist thing that Google has been doing with their websites ever since Plus came out, and you would like a dark version of that style for your Unity/Gnome3 desktop, Boje might work well for you.
All you need to do is download the theme, extract it into the ".themes" folder in your homespace. Then you can select it using the Advanced Settings program under the theme option on the side-bar. Set the GTK+ and Window theme to Boje. Once you have done this your computer will, more or less, honor this theme. Technically, this will only apply for GTK apps, but luckily most applications that you encounter are GTK applications. There is one hiccup that I've found, however. Firefox, for whatever reason, uses the system theme to color certain elements of web pages. Specifically, when using Firefox, input forms (e.g. text boxes, radio buttons, drop-down selectors) will have the background color of the system theme and, to make matters worse, tend to have the web sites text color. This means that many sites will have black bars on them which are actually search fields. You can type in them and they do what they are supposed to do, but you cannot see the text and they look ugly, which is more than a little bit annoying. To fix this (at least partially) Firefox allows you to insert overriding CSS style into the pages you visit via a file ~/.mozilla/firefox/yourprofile.default/chrome/userContent.css. If you edit this and insert a few styles that request that Firefox use white backgrounds on text fields this will make pages at least render readably. This is a general problem with dark themes and Firefox and really should be fixed.

## What About The Rest Of The World?

There is one issue with all this, while you might enjoy a dark theme, the rest of the world seems to have decided on bright backgrounds (something about thinking they look simple, or clean, or minimal). When you combine that with the fact that probably more than half your time is going to be spent in a browser window, you are going to be staring at and interacting with a lot of brightly themed interfaces anyway. But there is something we can do; we can extend our little hack for Firefox above to all web content and do it for Chrome as well using Stylish. Stylish is an extension for Firefox and Chrome/Chromium that allows users to fiddle with CSS styles (and perhaps more) on web pages that you visit. Think of it as a limited version of GreaseMonkey that is geared towards tweaking the visual style of pages. With Stylish installed you just click a button when you encounter a page that you don't like and find a style that fixes what bugs you. There are over 41,000 user submitted styles out there, but importantly they don't usually work forever (pages constantly change and the Stylish scripts will need to be tweaked). This means that the vast majority of these actually don't work completely. It is a process of trial and error, but it is easy to turn off malfunctioning style scripts. I guess I feel that they are worth the effort as they are exactly the best possible solution to this problem if they work, and sometimes they do. They allow you to do the following:

## When all else fails

I find that it is good to have a quick and dirty method to save your eyes if you have a non-GTK app or your Stylish script is broken or things are just generally not working. For this, I use Compiz's "Negative" plugin which you can configure using Compiz Configuration Settings Manager or ccsm. This plugin allows you to invert the video on a per window basis at the press of a key chord. I have bound "Super-n" to this functionality. This is far from ideal. This inverts all video which mean that all of the colors are screwed up when we use it, which makes video and images look like crap. But it is a useful fall-back method.

Update: Firefox can really look the part if you install the "Dark Bright-Aero" theme.

## Thursday, November 1, 2012

### No More Coursera Starter Kit

I just finished the latest Coursera assignment minutes before the deadline (a deadline that was extended due to Hurricane Sandy).  The assignment was not hard, but it certainly was time consuming.  Clearly I will be unable to put out a starter kit in any timely manner.  But, more importantly, I realized something when working through the assignment, this code that they have offered is a mess.  The act of porting this to stuff to Common Lisp would be a nightmare.  It is much better to rewrite the stuff.  The problem with this is that the actual assignments more or less require you to use the Octave code as it does things like sample from a random distribution.  It would be annoyingly difficult to guarantee that any code I wrote would sample the distribution the same way, especially since they use a built in MatLab/Octave routine.  I would have to research what PRNG Octave uses and likely implement it and make sure to seed it with whatever Octave is using.

The other thing I realized when working with the perceptron code is that writing starter code sucks for the programmer.  Writing code that has trivial pieces missing from it on purpose is not how I want to spend my time.  Creating teaching materials requires a crazy amount of prep-time and I don't care enough about education to spend all that time and, in the end, am left with nothing but a classroom exercise that cannot even be used for tinkering at the REPL.

This has lead me to the decision to stop my already feable attempts to maintain a Common Lisp port of the neural network starter code.  However, I still find that porting code is extremely useful in making sure you understand the concepts involved in the code.  I have always felt that once I could write the code for something, I tend to truly understand it.  So I will be taking the provided code from the course as a starting point and creating my own neural network related code, closely in parallel with the course, in Common Lisp.  I will not be leaving out the missing pieces as they do in the class.

I have decided that this won't really violate any rules of the course.  Looking at my perceptron code and the code that was released by the class organizers I doubt that any person would see one as derivative to the other if they didn't already know I was working off the other.  Further, I doubt that any person that couldn't answer the questions in the programming assignments using the MatLab code could look at my code and identify the pieces that needed to be ported back.  And it isn't as if executing the Lisp code will help them at all.  As I said, I doubt I could actually write code that would give the correct numerical answers for the homework even if I wanted to because of the MatLab/Octave heavy emphasis in the provided code.  I was able to just barely do it in the perceptron case and it probably took about half the development time to figure out how I would load the MatLab data and exactly match their form of the learning algorithm.

If you are following this work, I will probably stay on the same git repository for a while, but might switch to another more appropriately named one in due time.  Hopefully I will have a little toy neural network library to play with in a few weeks.

## Friday, October 19, 2012

### Neural Networks, Coursera, and Common Lisp

There is a course offered on machine learning using artificial neural networks offered at Coursera this "semester".  I am taking it and it is my first class I have taken with Coursera.  I have forgotten how nice it is to learn a new subject.

Besides the fact that artificial neural networks have always interested me, one of the reasons I decided to take this class is to become somewhat fluent in Python, which for some reason I thought would be one of the supported languages.  Turns out it isn't, but no matter.  Python is popular enough that someone will always port to Python.

I figure I might as well do the same for Common Lisp.  This is very late for the first assignment, and I make no promises that it will be more timely in the future.  I decided not to do a line by line porting of the Octave or Python code to Lisp because, well that's just not how we Lispers do things.  It is cleaner in its current form, but harder to prove correct at a glance.  Also, I will be using some packages that I have been working on but are not officially released, such as my zgnuplot library.  I have been using this plotting library with in-house code for a long time, but it is still pretty rough around the edges and the interface is still in flux (hopefully headed towards something a bit better).  But whatever, it's out there, feel free to do what you will with it.

Note to people that might fork this, the rules on Coursera (well, at least this course) is that you shouldn't post solutions.  So don't work off this repo, plug your solution into it, and publish it online.  You should probably work in a private branch that you don't publish to Github.

## Tuesday, September 25, 2012

### Stallman on Steam on GNU/Linux

In July, Valve announced that they were in the process of porting Steam and a handful of games to GNU/Linux. Shortly after, Stallman put forward this statement on the benefits of porting Steam to GNU/Linux. In this delightful and surprising opinion statement, he argues that the gains of video game loving people switching their OS, the foundation of all of the their computation, to a Libre alternative displaces the damage done by Steam moving to GNU/Linux. Now, we should all note that while Stallman can give his opinion until he is blue in the face and it doesn't necessarily mean anything. GNU/Linux is Libre Software and, by the definition of Libre Software, has nothing to say about what software can or cannot be run on the system.1 Be that as it may, Stallman's opinion does hold some weight for some for the community.

What is interesting is that this is a huge departure from the opinions Stallman typically expresses. From what I've seen, Stallman has never been one for compromise when it comes to Libre Software, even when that compromise would almost certainly result in benefit to the Libre Software community. As an example, you can listen to Bryan Lunduke and Chris Fisher's conversation with Stallman on the "GNU/Linux Action Show." Lunduke, who dominated the discussion, was looking for tips on how he could transition from a proprietary software developer into a Libre Software developer. In particular, he was looking for information on how he could do this and still support his family. Now, I'm not claiming that Lunduke is a programming superstar, or that Lunduke's software is so great that having it released as open source is a clear and substantial benefit for the community. I don't really know what effect his software would have for anybody. I will say that having a guide of how to transition to Libre Software development, even if only partially, would be a great help to the community, and reiterating the message of the Free Software Movement to any captive audience is likewise a benefit. Stallman's response was implicitly that that an independent developer (i.e. someone that makes their living selling digital copies of their software) simply cannot currently support themselves with Libre Software development. What he actually said, however, was that Lunduke should seek a new career. Specifically, his point seemed to be that comparing the viability of "for profit" Libre Software development to proprietary software development was a futile exercise since proprietary development is unethical and akin to "burglary". It might or might not make you money, but it should not be done for ethical reasons. While I agree with the gist of the statement, arguing with extremes like comparing a relatively minor restriction of people's freedom to something like burglary is frowned upon by most people. More to the point, I feel that many more minds can be won over when an understanding and softer, but still firm, voice is used. I invite you to watch the whole interview, as infuriating as it is. By the way, if you come off thinking that this Lunduke fellow was treated unfairly and that Stallman is a d-bag, you might watch the following week's episode where Lunduke throws a multitude of ad hominem attacks against Stallman, which will probably make you reassess where the d-baggery lies.

I bring up all of this so that the context is clear of what Stallman's opinion of compromise has been in the past. This opinion on Steam seems nearly 180 degrees out of phase from the standard Stallman opinion. It is not that I see this as antithetical to the advancement of Libre Software, quite the contrary as I will argue, but I do see it as not keeping with Stallman's usual position on the mixing of Libre and proprietary software, much less DRM laden software. It should not be lost on us that Steam is perhaps the most visible DRM peddling marketplace in existence today. This is the software that Stallman is tacitly ok with.

I wonder if this is a change in Stallman's strategy for promoting Libre Software usage. I wonder if he now sees more benefit performing a bit of social engineering rather than just leading by example. At the very least, for people that say that Stallman is not able or not willing to compromise or see the bigger picture (which I have been guilty of a few times) this should demonstrate otherwise. Whatever the reason behind this change of opinion, tone, or otherwise, I think that this is a good thing that hopefully we will see more of.

## A Libre Software ecosystem

I think that Stallman might be understating some of the indirect benefits of having Steam on GNU/Linux. Windows dominates the computing world because it has many users. It has those users because it has developers writing software. It of course has those developers writing software because it has all those users, who are willing to pay money. This self-sustaining ecosystem is something to be sought in the Libre Software world. Anything that can be used to bootstrap this, should be sought out. And yes, this means that people in the Libre Software world will need to get used to the idea of paying for what they want if the desire this kind of ecosystem.2 This is not difficult to work out. A developer only has so much time, some of this time needs to be used to make money to live, thus if they can make money developing free software there will be more free software. This is the reason I feel that the funding problem is the number one problem to tackle for the Free Software Movement.

The problem is, an ecosystem takes a long time to build up unless you are willing to pump vast amounts of money into it, and maybe it doesn't grow even if you do. It takes a long time to build, but what if you can have an already established ecosystem migrate to you? Valve has been relatively candid about their dislike of the direction that Microsoft is headed in, and who could blame them? There are some shaky decisions coming out of Microsoft and things are probably going to get much worse before they get better (if they do). I am expecting a pretty large exodus of users fairly soon.

The question becomes, where will these users go? Presumably, many will switch to OS X. However, as many have noted, Apple has some quite disturbing opinions (along with the track record to back it up) about what rights the users and developers of software for their iOS should have. When these limitations by Apple land on OS X, I expect that it will very likely discourage many Windows users from moving to that OS. This is made all the more troubling when you see Microsoft starting to tread down the same path Apple travelled as they seek to regain some of their lost market share and profits. These kinds of decisions by the major players in the field are a strong incentive for diversifying your product to a stable, yet non-restrictive OS.

Thus Valve decided to port Steam to GNU/Linux. I think the motivation of Valve's decision is just plain business sense. They want to buy some "Windows and Microsoft falls apart and not everybody likes Apple" insurance. This is a good way to hedge their bets on where their users will end up and ensure a longer life should Microsoft continue to make poor decisions. Further, they will have a substantive advantage over late comers to the GNU/Linux platform if it does indeed become popular amongst gamers. But the reasons for their decisions aren't what I am focusing on; I am interested in what effect we should expect for Libre Software.

## The benefit and harm of Steam on GNU/Linux

In a recent post (see the last section), I made a particularly Raymond-esque argument that the importance of having something be Libre is directly proportional to how important that software is to your life and livelihood.3 I concluded that proprietary video games are "ok" so long as they don't come with even a hint of DRM. Because of this "no DRM" standard, I will most likely never pay any money for any software sold via Steam, nor do I think anybody should for that matter, though I know and accept that many will.

The point is, once you accept this argument for proprietary games and proprietary stops being your cut-off for completely unacceptable, you start seeing shades of gray and you will most likely come to the conclusion that proprietary may be bad, but DRM is worse. It is a matter of degree. DRM in software marks that extra step past software that is merely proprietary but non-abusive to something which is, in fact, abusive. DRM is in its very essence not in the users best interest, the epitome of an anti-feature. In addition, DRM is very often more abusive than the original authors intended it to be, hurting "legitimate users" as well as copyright infringers. DRM is almost by definition the act of abusing the proprietary nature of the application in order to actively restrict a user's freedom in a very overt and measurable way. DRM is the actual boogie man, or one of them, that the FSF has been warning about for all these years. Here he is, in the flesh, right where we can all see him.

This makes me wonder if this move by Stallman is a bit more diabolical than it initially seems. I suspect that if there is one place where proprietary software is unlikely to "teach users that the point is not freedom," video games are it. To the average user, a video game is a piece of software that is the epitome of luxury. From a practical point of view, a video game is only as good as the art that goes into it, and from Stallman's point of view, Libre art is of much lower priority than Libre Software (which I agree with). What I think might be the ultimate result of DRM video games in GNU/Linux is a better appreciation and understanding of the main goals of freedom in Libre Software rather than an erosion of these ideals.

By letting the boogie man run around your town and terrorize your citizens, it often has the result of people increasing their support for the white knights that have devoted their life to protecting users from that boogie man. What better way to ensure a boogie man that will be noticed but not cause too much damage than to let in DRM, which is today basically synonymous with proprietary software abuse, but only in video games, which are hugely popular but basically synonymous with luxury? We may even get a few new white knights out of it. When these new GNU/Linux users boot up and find a wealth of software, in fact an entire operating system, all of which is devoid of DRM, lock-in, and excessively high pricing, they might, just might, start to associate proprietary with restriction, DRM, and "annoying to use" while, at the same time, start to associate Libre/Free/Open Source with freedom and, gasp, "easy to use." In fact, if we continue to produce these pretty impressive Libre video games, people might start to wonder what they need Steam for at all.

And if Steam isn't overtly abusive? What if people accept it and like it? You don't get this effect I describe, true, but you still have Stallman's original point: more users of Libre Software means more freedom for users. You alse get more people familiar with developing for GNU/Linux and, arguably, increase the pool of developers that might write Libre Software in their free time. More or less, this is a win however you cut it, but it might be a bigger win than Stallman lets on.

## The year of the GNU/Linux Desktop?

I know, this has been speculated about to death. But with MS actively imploding under user disappointment and Apple encroaching further and further on users' freedoms, there could very likely be a sizable of portion of the community that is just lost enough to jump to Ubuntu or similar.

The upshot of all this is that GNU/Linux is actually is a very good position to accomplish at least Torvald's initial dream, a sizable install base for Linux based systems on the desktop/laptop, and a large part of the FSF's goal, to get more people using more Libre Software. I know it has been said before, but right now I see a possible inflection point. If Microsoft continues to tank on their desktop/laptop software (which seems very likely), Apple continues to make their desktop/laptop computing environment more like the restrictive iOS, GNU/Linux gets Steam and game developers start to develop for GNU/Linux, and provided we find a good way to financially support Libre Software, there is a very good chance for a vast increase of the install and developer share. This increase will, at the very least, increase the number of users that have control over their own computers even if there will be a few, non-essential, pieces of software that will be out of their control due to DRM. Stallman is correct to encourage Valve's port of Steam to the GNU/Linux platform.

## Footnotes:

1 It should also be noted that prior to fairly new positions by Apple in their iOS, this would be a silly thing to even bother to say. Before that it would be considered financial suicide to do so. Thank you, Apple…

2 That is not to say that I don't see the great benefit in having Libre Software also be gratis, but these ideas are not totally at odds with one another.

3 I actually have walked back my opinion a bit. I got to thinking about my sister, who sees video games as a social tool. As a social tool, this now becomes an important part of her life. I would wager that there are many people out there just like my sister. It would be arrogant of me to argue that just because I find something to be "purely a luxury" means that this is true for everyone. If I wish others to respect my opinions on what is important enough to be Libre Software, I should respect that others might find something I think of as non-essential as important enough as well.

## Wednesday, September 19, 2012

### The Cost of Fuel

I saw this image on Reddit today and thought, hey, that's not right. All methods of transportation require fuel. For the bike it just happens that the fuel is food rather than gasoline or diesel. As it turns out, gas stations tend to ubiquitously offer two things, fuel for automobiles and snacks for humans. So, in actuality, the price not only completely inaccurate, we can actually calculate approximately what that price should be.

The price of the gasoline is set per gallon here in the US. In order to compare to food we need to compare in terms of a common quantity, a quantity that combines the energy density and efficiency of the engine. The quantity that really cuts to the chase combining all of these factors implicitly is the distance that you can travel. Specifically, we seek to compare the distance you can travel per dollar, which is actually how you should compare most energy sources, cost divided by some benefit measure.

A car can typically travel around 30 miles on one gallon of gasoline. The price of gas around where I live is on the order of 4 USD per gallon. This means that it costs 13.3 cents to travel a mile in your car.

A simple Google search tells us that a typical bicycle rider burns around 50 kcals per mile. At a gas station, with their premium on convenience, you can expect to spend something like 2 USD on a glorified bottle of sugar water which contains a total of 125 kcals of energy. This means that it cost you 80 cents to travel a mile on your bike.

Therefore, if you are purchasing fuel from a gas station and have the choice between travel by car or travel by bicycle, and care only about the cost of fuel to do so, it is much cheaper, six times cheaper, to purchase your fuel in the form of gasoline and drive your automobile around than to purchase food and ride your bicycle around.

But that's not all. The URL on the gas station sign tells me that this station is in Austria. They are on the Euro and the current cost of gasoline there is something like 1.4 Euro per liter, so I suppose the price at this station is 1.46 Euro for a liter of diesel. Another Google search tells us our 2 USD sugar water would cost 1.50 Euro assuming gas station snack foods cost the same as they do in the US. This corresponds to 37.5 euro cents to travel a km on bike to 11.4 euro cents to travel a km in a car, a ratio of 3.28, smaller but still favors travel by automobile. This means that the price on the sign should say something like 4.787 Euro.

## Tuesday, August 7, 2012

### Why I Have a Flattr Button

Update (Jan. 9, 2015): Many new services have come into vogue since I originally posted this, many of which are close enough to what I initially had in mind that they deserve mention here. Of these, Patreon comes to the forefront as nearly optimal, not exactly what I wanted, but pretty good. Also, I found that my idea that I allude to in this post turns out to not be such a good idea. I thought that developers would be happy to see people providing funding mechanisms to them, even if they didn't request it. I legitimately felt that, during the bootstrapping phase of the service, it would be a good idea to have bots crawl websites, repos, and mailing lists, train a model to determine how funds should be distributed amongst a project, and then distribute them via unintrusive email notifications to the developers. Turns out someone else was doing something similar to this and got seriously chewed out by several developers, so, that was a bad idea.

At the same time as I was rethinking my idea (which never found traction with people I pitched it to) Flattr changed something about their buttons so now they rarely show on my pages, and other stuff. So, yeah, nothing other than sending BTC is likely to work right now and I haven't gotten around to fixing it.

Flattr is a micropayment service geared towards funding artists (primarily bloggers) and community tech support. I am writing this blog as a hobby, as a sort of portfolio and record of things I have worked on, and as a motivation to learn new things and learn them to the point that I can explain them to others. Direct monetary gain has never come into the equation. I am also of the opinion that nothing I have written here actually warrants "Flattr"-y. However, perhaps someday something will. I would just let this silently go up and see if it reaps any rewards, but I have had the idea of funding in Libre Software and art in general on my mind a lot lately.

## The Future of Funding Art

There is a deeper reason behind the Flattr button and the Gittip button right beside it. I put the Flattr button up as I feel that they got a lot of things right when designing their funding service. I mostly support their method of micropayments, and just using their service is in some way a support of their business, as is writing this post. But it isn't just Flattr that I support, it is other services like Gittip, and to lesser extent Indiegogo, Kickstarter, and a slew of other payment systems that have popped up recently. Each of these services, as I see it, strike a blow towards toppling an empire built on false scarcity, built on copyright. This is a false scarcity because scarcity of art that can be represented electronically simply doesn't actually exist. Once actually produced, software, literature, music, and movies are not a scarce commodity. They are commodities which are infinitely reproducible at next to zero cost. The sole purpose of copyright is to allow us to integrate those perfectly copyable things into our current market system in which value is based on scarcity. This act of limiting reproducibility via legality using copyright and, more recently, via technology using DRM is the act of setting up a false scarcity. The fact that this is a false scarcity is the reason that people reject the propaganda campaigns to relabel copyright infringement as theft, and the reason that some don't see it as wrong at all.

I don't think that the proliferation of services like these is a coincidence or that the recent success stories are a fluke. Services like these are the first harbingers of a fundamental change to the market for these non-scarce things. This change will be so fundamental that we will probably not even say that there is a "market" for software, literature, music, or movies in a few decades as there will be something else, something based on pledges and donations that has yet to be completely defined. This won't be the death of art, quite the contrary, it will be a rejuvenation. It may, however, be the death of companies that make their money by limiting the freedom of users. How do I know this? I don't, I just have hope. But as Kay says, "the best way to predict the future is to invent it." I still feel that these services, or any service I have seen, haven't quite gotten it right for the markets I care the most about (Flattr and Gittip seem quite close but slightly misdirected). From my point of view, there is ample area for innovation in this "sector", and I (and a few friends) have an idea that is in the works which I hope to announce not too long from now, if things go well and someone doesn't beat us too it.

To make a long story short: even if you think all that is a bunch of nonsense, I hope that you at the very least find this little Flattr button to be much less intrusive than ads and much more easily ignored.

## Friday, July 20, 2012

### ICFP 2012 Post Mortem

Update: The results of the first round have been published. Our submission was eliminated in the first round (not a huge surprise) and we got 154th place out of a mere 221 actual submissions that the organizers actually received. We definitely could have done better, but this is not that awful especially considering some of the bugs that were found after the submission. Only the top 117 submissions advanced to the next round and we were not that far off. Ah, now I can't wait until next year...

After a very long coding session with very little sleep, ICFP concluded and my team did fairly well for my first time on my own and my teammate's first time all together. In the end, we did submit a program, which is more than I can say for some years that I have competed (and I do mean seriously competed and never submitted, not just tinkering around).

Whew, that was close…

I am always surprised at how long it takes to package up your program and submit it. I got our submission in at 6:59 AM, less than a minute before the deadline. It was quite the nail biting finish, to be sure.

## The Successes

I found this year's problem to be excellent. I never felt that I was completely lost for long periods of time and yet I never felt that I would completely explore all aspects of the problem, either. This years problem seemed much less mathematically deep than those of some other years, but it was interesting and very easy to get started with, which makes for a very nice ICFP contest problem.

The problem this year involved writing a program that played a little mining video game (here is a javascript version written by another contestant). A good submission would find an optimal path through the mine to collect as many "lambdas" as possible, avoid dieing (rocks falling on top of it or drowning), and find its way to the exit. This year's problem was very accessible. You could submit something immediately that would be a valid program (something that just printed an A for "abort"). Further, there was basically no barrier to improving that solution.

What was suspiciously lacking this year was the deep math or computation theory under it all. This seemed to me to be a basic "search as best you can until your time runs out" sort of problem. Then again, we didn't have a real computer science guy or mathematician on the team this year, just a web developer and a computational physicist, so I wonder what other people found that we missed.

Also, I am pretty shocked that the organizers didn't allow people to submit maps to try other peoples robots against. I guess it doesn't quite jive with the storyline, but you could make it fit. Adding direct competition is a good way to make a problem adaptive to the skill set of the contestants. Also, no leader board? Bummer. But these are minor things. The task was all in all very good.

The organizers were also excellent. I say that because I never had to contact them and I never felt (for long periods of time) that the problem description made no sense. Even when I felt (and might still feel) that the task statement was ambiguous, or in some cases just plain wrong, they provided ample examples to clarify at least how they interpreted the task statement's meaning.

My teammate was also excellent. He stuck with it through the entire weekend for very long hours and provided much help in thinking out solutions and strategies as well as hashing out the meaning of various parts of the problem. I only hope he found the contest as fun as I did. All in all a pretty good year for ICFP. Definitely the most fun I have had with ICFP in a long time.

### The Remote Development Environment

The collaborative editing tools were mostly successful. There are some annoying (and productivity hurting) bugs that we hit using Rudel, usually when disconnecting and reconnecting later, but for the most part it worked quite well. The lack of graphics were a real detriment for me. I could have set up a visualization program very easily if there was a good way to use it over the network, but instead we fought with rendering the maps to Emacs buffers. My teammate suggested we use Flash for visualization, something he has a lot of experience with. While I was reticent of the idea at the time, due to my aversion to using flash in general, I now think that this would have been a good course of action.

In the end, however, I very much enjoyed text based setup we actually put together. This allowed us to have meaningful printed representations which are invaluable for debugging a program, something that we would have implemented anyway. After three straight days of watching our robot wander around a mine I feel like I have been playing Nethack for the last 72 hours, battling a wily rust monster. On the right is an example of our robot performing a search on a mine I made up using a particularly bad search heuristic. Each frame is a system state the robot is considering in the A* search.

In my opinion, the usage of Rudel made for much less "work off the reservation", basically people working on some aspect of the problem that really isn't that important to the group. In addition, the fact that we were working on the same image gave a certain urgency to changes that temporarily broke functionality. I still think this is a net positive for development of this type, but I would also like to try it more to determine if it truly is.

For a project like Double Fine Adventure, perhaps it is not too far off to think that \$550k was due to the GNU/Linux support, and perhaps \$350k for Wasteland 2. To contrast, one wonders what to expect for another project I would have been very interested in, Shadowrun Returns, which has announced that they will not have GNU/Linux support. Are they missing out on one sixth of their possible funding? Is the cost of porting to GNU/Linux more than that lost funding? Update: Shadowrun Returns has announced that if \$1M dollars is reached, there will be an effort to port to GNU/Linux after the release. This is pretty nice and while it would be great to have Linux support promised now and without the post release delay, I'm pretty sure that \$1M will be reached and any support is better than none. Yay! I won't pick on Al Lowe here, as budget is his primary concern right now, and it is too soon to tell what is going on with Jane Jensen's project. I wish you both the best and I'll be watching for other (hopefully new) titles on Kickstarter. I will say that I am kind of over LSL (much more of a Space Quest and King's quest fan) but if there was a GNU/Linux version, I'd throw \$15 at it to support it. So, here is the point. I think these projects should be supported. Go do it, now. But I also think people might want to start considering GNU/Linux as a deployment platform, and not for the novelty of it ala Id Software, but for the profit of it. Now, I realize that game developers are not reading this blog (I know this is true because no body is reading this blog ;) but maybe if you are reading it, and you run into someone that does make these kinds of decisions, some of these ideas will leak through during your conversation. Also, since I don't want to post on this stuff again, a quick note. The Humble Bundle started as a promotion where the games were released as FOSS after a certain goal was reached. This is no longer the case or this goal is very high. I guess I just want to relay my own data point to the world on the value of libre source. I paid \$30 for the first bundle after I heard of the possible source release. Now I have dwindled to around \$5 for each. Just a word of advice, releasing the code as FOSS is valuable to people and people are willing to pay more for it. #### Uh, did you forget about Libre Software? Oh, right… Well all of this stuff happening in our little nook of the world of video game development is great, but not exactly perfect. However, I have a differing opinion on video games than the FSF has. To me, closed source is 'okay' for DRM-free, non-mission critical software. Contrast against something like a computer algebra system that you use for your job, a financial books program that you use for your personal or work finances, or an operating system where you run everything. Having these be closed source means that you have relinquished a great deal of knowledge of how useful your work is, and correct and accurate your financial records are, and how safe any of your private data is. Having these be open source but non-Libre means that you have relinquished control over these things, but at least you know where you stand. In an ideal world, everything would be FLOSS and we would have come up with a widely applicable funding mechanism, but this isn't an ideal world yet. If there is going to be something non-Libre in the mix, I suppose that a video game that is fundamentally a luxury, is the best one to have, but only so long as there is no DRM attached to it. This is a big 'but' and I cannot stress this enough. If there is one place where we have demonstrative proof of the abuse of proprietary software, year after year after year, it is with DRM for video games. So, let me reiterate. Libre games would be a boon for gamers. I would like games better if they were Libre, all else being equal. But, the importance of a particular piece of software being Libre Software is directly proportional to how much of my life depends on it. For video games, this is sufficiently unimportant to the point that I feel that it is okay to support DRM free video games, particularly if there are fringe benefits like attracting more users, developers, and hardware vendors to GNU/Linux. ## Saturday, March 31, 2012 ### Running a Command Whenever a File Changes Edit: I actually had to republish this as I didn't understand the Blogger interface. Because of this, I added a new section where I combine this into a script that you can actually use. Enjoy. Every once in a while you probably would like to have a command run every time a particular file changes. I know I did when I was writing some Latex based presentations with Beamer and was using the latex to dvips to gs chain of commands to compile the document into a PDF file. I came up with a way to ease these steps by detecting changes in the document's DVI file that Emacs writes for me. I later thought of a couple of other methods and figured I would put them up here in case anybody ever needed something similar. I included all of them as you never know when you need to put together a hack on some old or limited system, or just a system that you don't control and thus cannot install new software. I decided that the interface of my little script should be to specify a file to watch, the watch file, and a command to run when it changes, the on change command. The on change command should take one argument, the watch file. ### Watch File Modification Times In this version we look at the watch file modification time and compare it to a reference time that is updated every time the watch file changes. This is definitely a cheap way to find these updates. The easiest way I could find to check the modification time of a file was to create a temporary file which acts as a time stamp. We just check which file is newer and touch the temporary file any time we want to update the time stamp. #!/bin/bash watched_file=$1
command=$2 # First we need a temporary file to test against marker=$(mktemp /tmp/on-change-XXXXXX)

# We have to trap the EXIT signal as we want to clean up our tmp file
trap "rm -f $marker" EXIT # Loop until killed while true do # If the watched_file is newer than the marker... if [$watched_file -nt $marker ] then # Wait for a bit to make sure that the file is done being modified sleep 1 # ... run the command ...$command $watched_file # ... and reset the marker touch$marker
fi
sleep 2
done


One thing to note is that this script, like many scripts, has a race condition in it. If it happens to detect a modification of the watch file but the modification is not yet complete, it will trigger the on change command and then probably get an error from it, then detect another modification the next iteration. This will continue until the program modifying the watched file has completed its work. This is bound to happen for short polling times (here we use two seconds) or long running programs that modify the watch file. This is the reason I added one second delays after the modification time based test. The delay is to try and give time for anything that is currently underway to complete. It helps, but of course fails to help other times.

A possible problem here is that it is easy to trick this into doing more work than it should. The watch file might be repeatedly touched but never actually modified, which would lead to unnecessary execution of the on change command. The key is to note that the newer modification time is a necessary but not sufficient condition for the file to have changed. It leads to false positives, but is very cheap.

It should be noted that, in the case of compiling Latex documents, while this can happen, it is a pretty rare event. But, if we are building a general purpose tool, it is something we should worry about.

### Watch File Hash

How can we eliminate the reprocessing of identical files? File hashes.

The next technique is to poll the file for actual file changes. We don't need to keep an old copy of the file, or anything like that, we just keep track of the file's old hash value. Every two seconds (plus the time it takes to hash the file) we compute files hash and compare it to the old file's hash. If they differ, we run the specified on change command and update the stored hash value.

#!/bin/bash

watched_file=$1 command=$2

# Compute the reference hash value
hash=md5sum $watched_file | cut -f 1 -d\  while true do # Grab the files hash newhash=md5sum$watched_file | cut -f 1 -d\ 
# If it has changed...
if [ $newhash !=$hash ]
then
# ...run the command...
$command$watched_file
# ...and record the new hash
hash=$newhash fi sleep 2 done  This has the advantage that it never needs to run the on change command unless there is an actual change in the file. This means that for an expensive on change command the method works very well. However, for very large watch files the hashing becomes unnecessarily expensive. However, the race condition is still there and this time it is worse. Whereas in the modification time test method we could try to give the modifying process extra time to complete, this time we don't have that option. We just have to blindly try again until we get it right. ### Use INotify As is often the case, you find something that is a better solution than what you have hacked together after you have long since found a good enough solution to your problem. This is the case with the me and INotify. The INotify kernel facility was designed for just the problem I was attacking. It provides, via a kernel interface, a way to hook into the file system and receive notifications on events such as reading from, writing to, opening, and closing files. If you are on one of the mainstream distros and have been keeping things even remotely up to date, you probably have an INotify ready kernel, but might not have the shell tools installed. We will be using inotifywait, which blocks until a specified file system event is triggered. In this case, we are interested in file modification events, so we will pass the option -e modify to the program. Just a note, this is taken almost verbatim from the inotifywait man page. #!/bin/bash watched_file=$1
command=$2 while inotifywait -e modify$watched_file; do
# Wait a bit, in case the modifier is still working
sleep 1
# Then run the command
$command$watched_file
done


This is much simpler than the other methods. A real strength here is that this is short enough that it really doesn't need to be a script at all. You can just memorize this. This is definitely a pretty good version, but it still has that race condition. So, one last version, where we will wait for the file to not be modified for a few seconds before we run the command on it. We can do this because inotifywait provides a timeout which exits with a 2 if it did timeout.

#!/bin/bash

watched_file=$1 command=$2

while inotifywait -e modify $watched_file; do while [ 2 !=$(inotifywait -e modify -t 1 $watched_file \ 1> /dev/null 2> /dev/null; echo$?) ]
do
echo waiting...
done
$command$watched_file
done


This catches most of the race condition issues, I think, but the price you pay is that you have to wait at least one second after the file has been modified before the file can be processed by your script.

Of course, without a proper synchronization mechanism, which needs to be agreed upon by both programs and thus is largely incompatible with the shell idea of small self contained programs, you will never get rid of this race condition. It is firmly up to the user to ensure that two different programs are not accessing the same file at the same time (actually, I think we can go a long way towards solving this by using lsof and checking if the watched file is still open. I don't have time to explore this but lsof $watched_file seems to be promissing). ### Combine The Methods We can have the best of all of these methods by combining them into one monster script that checks for changes using INotify (with a modification time check fall back) and then confirms that the file has actually changed by comparing the file hashes. But I wont bother because this is already too long, both in words and time spent typing and editing. Update: here it is. I changed the interface, now you specify a file and the complete command you want to run on change. #!/bin/bash watched_file=$1
command="${@:2}" # We start with a bogus hash hash=null if which inotifywait then function wait_till_unchanging() { while \ [$(inotifywait -e modify -t 1 $watched_file \ 1> /dev/null 2> /dev/null; echo$?) != 2 ];
do
echo waiting...
sleep 1
done
}
else
function wait_till_unchanging()
{
# Wait until there isn't a change for delay seconds
touch $marker cont=1 while [ 1 ==$cont ]
do
cont=0
sleep 1
# If the watched_file is newer than the marker...
if [ $watched_file -nt$marker ]
then
# reset the marker
touch $marker cont=1 fi done } fi if which md5sum then function if_changed_run () { # Grab the files hash newhash=md5sum$watched_file | cut -f 1 -d\ 

# If it has changed (this is always run the first time as $hash is null)... if [$newhash != $hash ] then # ...run the command...$command

# ...and record the new hash
hash=$newhash fi } else function if_changed_run () { # ...run the command...$command
}
fi

if which inotifywait
then
while inotifywait -e modify $watched_file; do wait_till_unchanging if_changed_run done else # First we need a temporary file to test against marker=$(mktemp /tmp/on-change-XXXXXX)

# We have to trap the EXIT signal as we want to clean up our tmp file
trap "rm -f $marker" EXIT while true do # If the watched_file is newer than the marker... if [$watched_file -nt \$marker ]
then
wait_till_unchanging
if_changed_run
fi
sleep 1
done
fi


### Using It To Automate Latex Builds

I said I wanted this to make compiling Latex documents easier. In order to do that you just use something like:

on-file-change presentation.dvi dvipdf


Update: with the new interface, it looks like this:

on-file-change presentation.dvi dvipdf presentation.dvi


Of course the best method would be to figure out how to make Emacs run this post processing for me. However, I have so far failed to figure that out and this little tool is applicable to more than just this scenario anyway.

Update: I did find out how to do this the right way in Emacs. Turns out there are a lot of different ways and not all of them work. For instance, the first answer on that page doesn't work for me. I use AuCTeX, which means that you can temporarily change the Latex compilation mode to use pdflatex by using "C-c C-t C-p" in the buffer or permanently set it by adding (setq TeX-PDF-mode t) in your .emacs file.