December 20, 1999

Thoughtful Programming and Forth: Conclusions

by Jeff Fox

Conclusion

Novice programmers, or commodity component programmers in some languages are given a problem and start right in writing (or pasting) code. More experience programmers know that time invested in understanding the problem and designing the solution is more important than jumping into coding. The master programmer knows that this stage may deserve being done more than once or recursively. If you cobble together megabytes of code and then start testing it you would never consider starting again. You already had to produce 1000x more code than you needed to, would you want to do it again? If you carefully construct some well thought out code to match the problem at hand more thinking or a little more recoding may be more fun and more productive than the first try.

So if you’re new to Forth try it out. Perhaps you can find some way to get that exhilarating 10x improvement in your programming. If you have been doing that for years remember how much fun it was when you made that improvement and consider that it could be fun again. Try for that second 10x with an open mind. OK, so you have been doing all this for years and your programs are about as small and simple and fast and clean as you think is possible. Great, but don’t stop there. Unless you think you really have reached the end-all of computing be willing to try other things, experiment, look for stuff you hadn’t noticed before. If on the other hand you don’t care about these issues and are delightfully happy to do your sentence as a replaceable commodity programmer and you find learning painful then none of this is applicable to you. It is only for people who want smaller, faster, cleaner, clearer and better programs and more personal satisfaction.

As Chuck has said one of the rewards of thoughtful programming is the satisfaction of a job well done. If you enjoy programming you probably enjoy doing it well and you may not enjoy being told to crank out low quality code because it seems like a good idea to some bean-counting manager. It makes you wonder how many times programmers expressed concern and were forced to put Y2K bugs into programs against their will by short sighted managment looking at the quarterly budget.

One thing I am sure many people would think is that the techniques that Chuck uses are examples of individual genius and individual effort that they are not going to be applicable to typical problems in computing. I know that Chuck would not agree with that nor do I. I have seen too many other people do it, and on teams. I worked with Chuck through my own company UltraTechnology and I worked as director of programming at the iTV Corporation with Chuck. There I was able to examine the relationship between team effort and programming environment and style. I was somewhat suprised to find that a team of Machine Forth programmers using Chuck’s style of problem solving and coding combine with a management effort to guide the team worked very well. It really didn’t appear that the language or style of coding had any relationship with team coordination efforts other than the relative number of bugs that different environments introduced. The Machine Forth programmers had no problem sharing problems, sharing code, and having their contributions to the effort work in harmony with the other programmers. It confirmed my experience that it is not the language but the way the team interacts that is the biggest factor. However if the coding style introduces bugs they are harder to find in a multiprogrammer project. The Machine Forth programmers consistently delivered very stable high performance bug free code on schedule. It was also my job to train these programmers on Machine Forth and I pleased to see that the techniques were easy to learn and easy to apply.

Another thing I am sure many people are going to think is that the techniques that Chuck uses is useful for dealing with small problems but not large one. However the largest and most complex, not to mention most expensive, software that I have encountered were VLSI CAD programs. I am sure people will equate the incredibly small size of OKAD and its incredibly fast code with it just being a trivial problem. Chuck’s extreme optimization of the software only shows that his approach allowed him to write a suite of application programs that perform the same functions that he wanted to use in those huge expensive VLSI CAD programs. His ability to shrink a number of megabyte sized application programs into his set of a number of kilobyte sized is not an isolated case. We had a set of programmers who consistently wrote bug free 1K sized applications in Machine Forth. In one case we had one ANS Forth program that was 100K rewritten to 1K in Machine Forth and made 1000x faster by someone applying these programming techniques.

So why are Chuck’s ideas about computing so unpopular? Well people tend to equate them with twenty five year old Forths rather than seeing that he has significantly changed his approach to Forth several times over the years. What is funny is that they will say they have no interest in what Chuck is doing because he is still doing what he was doing twenty years ago. Chuck is doing Forth without ‘C’ libraries linked in and with BLOCKS and with tiny non-ANS style Forth. Chuck would say that they are the ones who are twenty years behind and are still doing what he was doing twenty years ago, they just created a poor committee standardized version of where he was twenty years ago. So it is almost funny that both views see the other as twenty years behind.

People get very offended when Chuck just plainly says that their code is 10x bigger than required, and those are the Forth experts! The people doing ‘C’ or Forth in ‘C’ look at what he said and see those 100x numbers. They feel very insulted and sometimes claim that Chuck called them an idiot just because he reported the result of a benchmark or the time he spend coding something. They would rather refuse to believe the numbers or even refuse to look at them because they might shake up their ideas about computing. If you are selling megaForths then Chuck’s ideas that Forth only needs 1K and more than that is most likely unneeded fat is not very attractive. If you are getting paid to teach people how to write ANS Forth programs you might not like Chuck advising people to try something else. If you never saw the original 10x and don’t think that other people really say it you are not likely to consider other hidden potential 10x factors.

Chuck’s ideas are really in the face of many of the things that are being taught in computer science. They are in the face of all the folks who’s computer software has been expanding about as fast as their computer hardware for years or even sometimes falling behind. They are not applicable once you find yourself buried in self-imposed complexity. They won’t help you if mix and match them too much with the conventional ideas.

While most of the Forth community has been working very hard to make Forth more like other languages to get it to fit into the niche the world has for it Chuck has been trying to go in an opposite direction. Rather than water down or dumb it down to look everything else Chuck has continued to make it smaller, simpler, faster and more productive in iteration after iteration. Just as Forth was in the face of conventional programming Chuck has chosen to not do what he thinks most everyone else is doing, making Forth look and work more like other languages. He feels that are plenty of people doing that. He wants to try to make Forth smaller, simpler, faster and more Forth-like again and again. That is in direct opposition to most of the Forth community who want to agree to set in stone the way things were done twenty years ago and then extend Forth further and further.

Neither effort seems to have done much for Forth. In this period the size of the Forth Interest Group dropped steadily, leveled for a bit, then fell off again. Today I think c.l.f is a larger Forth community than FIG. It is a shame too in the sense that FIG asks the experts to give presentations and then ask them questions. Usenet offers equal time to newbies, green belts, black belts and masters and the experts are more focused on exposing the fringes of Forth to people by endlessly debating ridiculous ways to break the standard with some of the wierdest code any of us have ever seen. I really wonder how anyone would get started with Forth today and how much of that original 10x, will be available to them.

Final Thoughts

Chuck prefers the use of seven keys, cursor and function keys rather than a mouse for precision of control. He uses a full keyboard for writing Color Forth scripts with function keys assigned as color change tokens. His GUI in OK uses the full graphics screen in a number of different modes of operation rater than having the look of popular GUI with resizable, movable, layered windows on a simulated desktop. I prefer the mouse driven interface to the seven function keys but the interface is still left, right, up, down, and a couple of buttons. I have said for years that something that looks to the user like a popular GUI with resizable, movable, layered display windows and a simulated desktop only takes a couple of K of code. Chuck has demonstrated that a windows accelerator in hardware only requires a few transistors. These things can be simple, fast, and mostly painless. The application of Chuck’s methods fits well when creating a small simple GUI or OS with efficient implementations of the abstractions that the application demands.

Chuck’s idea of the future is more custom silicon and machines that are efficient at what they do. All machines will not need Windows ™ or Unix or Forth for that matter. Chuck likes the abstraction of source code being interpreted in embedded applications. He has even joked that perhaps after Y2K it will be mandated. With a tiny 1K Forth or Forth in hardware machines would embody underlying abstractions used by programmers. Chuck’s methods apply well to a team of programmers setting the abstractions that they need. Chuck has said that he would enjoy seeing a small project be funded to have a small team of a few people knock off something resembling the modern GUI desktop and set of applications in Forth on a Forth machine that does everything useful but that is a thousand times smaller and simpler than the alternatives people have now.

A tiny piece of silicon can contain hardware that performs a particular task with incredible efficiency and with a tiny general purpose central processor. One advantage of these machines will be that software will support only the devices found on chip. You can have all the special cards, video cards, analog cards, network cards, lcd support, gigabit fiber links, popular I/O interfaces etc. and for a given device there only needs to be one set of drivers to support the hardware. There is no need to support many extra layers of abstraction or inefficiency between what needs to be done, what the hardware can do, and what the software does.

After of number of years of working in these environments I must admit that I feel that I have become very spoiled. I am used to having fun solving problems quickly and feeling very productive. I have a strong sense of satisfaction with the problems that I have throught through well. I enjoy showing other people how easy it is to do things this way. I recall how when I worked as a consultant to big companies on all kinds of computer with all kinds of software that I enjoyed solving the most complex and involved problems and I think how different that world was than one where the problems are small ones and I can get so much more done. I can also understand how most people look at Chuck’s methods on the surface and think that it just doesn’t apply to what they have to do. Chuck’s methods wouldn’t solve all of the problems that they have. Their problems are often related to different methods and Chuck strategy is to avoid most of the problems other people must face to be more effective at solving the problems he wants to solve.

Truth Bleeds Red 2018