Programming made a major leap from the pocket-calculator style "data-in" into "a program" which produces "data-out" when computers became able to refer to themselves, so to speak, via the concept of "reference".

Initially conceived as having access to a memory address (and still the case in programming languages such as C/C++), this is now more commonly referred to as a handle, in the case of data, and as a collection of all sort of funny little things like delegates, method-pointers and so on in the case of code.

A data reference allows us to work with the address of data about which we either know nothing (weakly typed) or everything (strongly typed). The weaker the typing, the greater the flexibility and the bigger the risk of losing track of what we're doing (a polite description of a bug).

A code reference allows us to work with the address of some executable code about which we either know nothing (weakly typed) or everything (strongly typed). We can even jump into it and see what happens (see previous comment on flexibility).

Object Oriented Programming has allowed us to start thinking of both in terms of knowing certain, different, things about our reference targets at different times, according to our need. This is a tremendously powerful programming advance and so is, not surprisingly, one of the principles of software engineering all by itself (#5).

The basic principles, however, are still important.

Although template/class/code libraries have provided ready-made mechanisms for storing and sorting lists of data, there are times when building your own linked data structures results in clearer code. Genealogy provides a good example. Although you would want a master list of your population keyed by their name, say, you would want each person to directly link with their relatives rather than force you to keep looking up cousin "Jane" and her brother "Harold" and so on.

(Someone may argue against me with this, and unless performance is an issue (which it frequently is) there is an argument for not trying to maintain multiply linked lists (because it's an error-prone headache)).

Although high level programming constructs have largely removed the need for function pointers the "goto" syndrome persists in a way which makes me think it must be tied to an ancestral memory. C#, for example, wraps a function in a class in a weird "transparent" way and even C++ has pointers to class methods (an OO term). Then, of course, there are "exceptions", a goto by another name (IMVHO) with a transparent function pointer passed in by way of an exception handler.

Whether we like it or not function pointers seem to be here to stay in some guise or another so, again, it's a weapon worth keeping in a programmer's arsenal.

Richard

The Principles of Software Engineering: start previous next

The most basic principle is the one with which software engineering began, i.e. the idea that you can solve a particular problem by performing a sequence of steps. Computers owe their very existence to the fact that they can do this incredibly quickly.

Some programmers never really get beyond this. Others go so far beyond it that they need to be reminded (occasionally ) that the most elegant solution to a problem may well be a simple sequence of steps, declarative/functional programming notwithstanding.

Hot on the heels of the processing concept came the various flow of control programming directives (if-then-else, while-do, etc) including the infamous goto, which some programmers (me included) think is the worse thing in the world and others couldn't live without. Nothing very new has appeared since the introduction of exceptions, a concept still trying to find its feet between those who (like me) use them extremeley sparingly and others who think you should use them all the time (for the record, I use exceptions in abort situations and test-harnesses and rarely anywhere else).

Multi-tasking, which is where more than one process takes place in parallel (not necessarily on the same processor), has a strange relationship with software engineering. In many cases it isn't a software engineering issue at all, more a case of hardware configuration and software deployment. Multi-tasking becomes a software engineering issue when an engineer has to make the decisions on how to partition his software into asynchronous (i.e. independant) modules.

Although multi-tasking has been around for at least 25 years, it is relatively unexplored as an engineering technique. Separating functionality into asynchronous modules communicating to each other via pipes is easy enough, and occurs naturally at the boundaries between, say, an application and a print-spooler, or an application and an operating system (like Windows). Seaparating functionality at a fine grained level is also easy to do - i.e. everything talking to everything else in an asynchronous manner. It's not very efficient, though. The problem is finding the optimal middle ground between these two extremes, and this is totally dependent on what the program is actually doing. Where this partitioning happens, it tends to be done on the basis of module-cohesion - i.e. grouping together everything which is related in some fashion. Multi-tasking, however, is all about performance rather than elegance of design. Module-cohesion may not be a bad first attempt but something more intelligent is required here.

Well, single-task programming is hard enough without adding this particular complication - at least, for now.

It is also worth mentioning that multi-tasking makes a significant appearance in low level programming, where the behaviour of peripheral devices provides a natural multi-processing environmnent (and I include in this keyboards and mice). There is a little bit of technique here to do with polling-vs-interrupt, safe-areas, locks and semaphores which anyone involved in this type of work will have to be familiar with. Generally speaking the order of the day is to try to minimise any processing you do inside an interrupt (i.e. when you've stopped your main process in order to do something else). Failure to do this could land you getting into all sorts of horrible real-time bugs (been there, done that, have the scars to prove it ).

Richard

The Principles of Software Engineering: start next

The bug reports are coming in and it's serious. Your software ships in a few days or, worse, it's already out there. The CEO has pawned his jag and the only programmers who understand the code have either died of malnutrition, exposure, or gone back to live with their families in the Amazon Basin. You, the Engineering Manager, have placed your neck and certain more delicate parts of your body on the chopping block. What do you do?

1. Make sure it can't get any worse. If you've got some people working on the problem ask them to produce the current best solution they can and make sure it's backed up and reproduceable.
2. Understand that you have a commercial problem much more than an engineering one. At this stage of the game it's going to be all about making compromises between short-term, mid-term and long-term revenue. You may fool some people into buying now even though you're in a mess, but you risk them never touching you again with a barge-pole later.
3. Try to establish whether you actually know everything that is wrong with your product. Ideally you would have an extensive test-harness which would run reasonably quickly and which you could add to if necessary. If you haven't, you're going to have to accept the risk that any changes you make could introduce bugs somewhere else which you may not spot. It's probably a good idea to start building a test-harness now as a mid-term solution if you have the resources.
4. Evaluate all known bugs for their commericial sensitivity and risk. I'm assuming none of these are trivial bugs - in fact, they're mainly "What on Earth" bugs (see here) - so fixing them isn't necessarily possible in the time scale (in fact, you've no idea how to fix them, so you've no idea how long a fix would take).
5. Tweak. This is especially dangerous if you've failed on point (3) above, but it may be your only option. Ask your engineers to use their intuition given what they know about the code to suggest little changes which may magically make the situation better (i.e. improve the bug ratings that you've made in step 4). Since you are, largely speaking, poking in the dark, make sure you can undo any changes which turn out to be disastrous.

6. Start trying to improve your understanding of the code by:

   1. Making sure all your engineers have note books and asking them to write things down that they discover. In particular, engineers do not like to digress when they spot something whilst working on something else. With a little notebook by their side, they can write it down and come back to it later.
   2. Trying to discover where the boundaries of your code are - i.e. sections whose interface is clearly defined and which cannot side-effect or be side-effected from other areas. Work on the smallest / simplest first.
   3. Tidying up your use of spaces, tabs and new-lines. Use white space as a means to make the code as easy to understand as possible.
   4. If you have a safe means of doing so, renaming variables, functions, and so on into more descriptive names, even if these are really long (like "check_date_field_in_record_and_update_next_appointment_date_if_in_the_past"). If you can say it, totally accurately, in just a few words, then great, but accuracy now is more important than brevity. This will hopefully have a knock on effect that may start to highlight where mistaken assumptions have happened in the code.
   5. Reviewing comments. Get rid of any that don't make sense. Get rid of clutter that isn't affecting the functionality of the code. Use comment patterns (like rows of stars(*)) to highlight particular parts of your code (like all functions). Like white space, use comments to make it as easy as possible to see what is going on.
7. Simplify (dangerous without step (3) above). Get rid of repetition using functional and object abstraction as safely as possible. Try to reduce the size of your problem without making it more complicated.
8. Fix the bloody thing. Or throw it away.

Richard

Engineering is one of two types of Programming, and Programming is only one of the many disciplines which are needed to deliver IS/IT (i.e. an Information System based on Information Technology, or more simply a Computer based Service like an airline reservation service or a web-based service like Amazon).

To re-iterate briefly (this), Engineering is the process of software construction at the point where the problems posed by all other sections of IS/IT have been overcome. Engineering's challenge is solely the complexity of the application itself, and it is precisely in the area of complexity management that an engineer's skills must be spent. Failure means the program at best no longer works as intended and at worst fails beyond recovery.

Whereas all other areas of programming and IS/IT have moved with great rapidity as new technology has developed, sometimes genuinely as advances brought about by Research and Development (R & D) have made new things possible, sometimes falsely as the market is fooled into believing something "new" has appeared which is only a repackaging of something "old", the principles behind Engineering have been slow in coming but significant on arrival.

This isn't the "sexy" end of IS/IT - far from it - and yet it is by far the most challenging. It is also where software projects fail. The answers to the problems posed to engineers cannot be read in a book. Hand any project manager a failing software project and there's no guarantee that he'll not be able to do any better than suggest the whole thing be thrown away. Although the principles may be few and easy to learn success comes down to their application and to a few key skills mainly in the realms of care, diligence and a drive towards a kind of "perfect simplicity" sometimes disparagingly called "elegance".

Richard

The Principles of Software Engineering: next

There are Technicians and there are Engineers. I have never found anyone who could be both, since they require totally different types of people. Engineers used to be more common than Technicians but now it's the other way round. In any software project you're going to need both, though the exact mixture depends on the project

Technicians love technology. They like to find out how things work - anything from chipsets to graphical interfaces to miriad applications which can be glued together and leveraged in order to build their final systems. Technicians in a project work at the interface between that project and the outside world. They can build prototypes to exercise whatever sits at the other side of that interface in as rigorous a manner as you like. Once a technician has found out everything there is to know about some piece of technology they like to move on to something else. These kind of people are vibrant and energetic, always up on the latest *everything*, and are generally at the more youthful end of your programming workforce.

Engineers have more of a love/hate relationship with technology. Engineers like to build ever more complex systems once the hassle of the technology interface has been sorted out. Engineers will be up on the latest programming techniques, but these move with geological slowness compared to the latest flavour of spreadsheet, editor or what have you. An engineer is only interested in learning something if it is *truly* new. An engineer is much more interested in developing ways to write software of increasing sophistication whilst maintaining a sure grip on it so that it doesn't start running away with bugs. These kind of people are careful and meticulous, always thinking about what they're currently working on, and are likely to be at the more experienced end of your workforce.

There are also two "swindled" versions - one of each:

A Pretend Technologist only ever learns enough about technology to know more than his employer or client in order to sound knowledgeable when he actually isn't. This sort of person moves *very* rapidly from one technology to the next, not only because he cannot get a grip on anything beyond the headline level but also to keep running away from anyone who might be following along trying to expose the swindle.

A Pretend Engineer works and re-works the same code endlessly, adding more and more levels of documentation and "safeguard". He never wants to finish what he's actually doing because he is actually incapable of writing working code.

Richard

(an engineer)