Engineering is, at its core, the process of transforming information into designs that solve problems.

This has a number of implications:

1. An engineer with access to lots of good information is able to produce better designs.
2. Having a lot of data doesn’t help if you can’t find the data you’re looking for. Therefore, engineers should take an interest in information storage, search, and retrieval technology.
3. The engineer who is able to automatically transform information into designs is much more productive, and will make far fewer mistakes, than the engineer who does this manually.

Points 1) and 2) are a subject for another day. Today I would like to write about point 3).

The first dirty secret of the engineering profession is that a professional engineer, working in industry, spends at least 50% of their time manipulating data in Microsoft Excel.

• A mechanical design engineer, working on a wastewater treatment plant, spends hundreds of hours developing:
  • piping schedules (spreadsheets)
  • valve schedules (spreadsheets)
  • machine operating point calculations (spreadsheets)
• An electrical engineer, developing a power system simulation, spends hundreds of hours:
  • Marshalling cable information for input to the simulation program (spreadsheets)
  • Collecting protection relay settings (spreadsheets)
  • Analysing the simulator output data to glean insights into the power system (you guessed it; more spreadsheets)

The spreadsheet work usually involves:

1. collecting data from a multitude of sources - manufacturer’s data, design guidelines, standards, and so forth;
2. transforming the input data into design output values by filtering, sorting, and calculations.
3. presenting the design output values in a form suitable for the users of the data; i.e. drawings, purchasing schedules, and so on.

A typical example in electrical design is deciding how big a cable needs to be.

1. The input data includes the load requirements, cable manufacturer’s data, and the Australian Standards for electrical design.
2. The input data is put through a spreadsheet (or specialist software) that considers the design requirements, the resistance of the cable, the minimum requirements of AS3000, etc. to select a cable size.
3. The results of many such cable sizing calculations are collected into a spreadsheet called a cable schedule, which says how big each cable is, how long it is, etc. The cable schedule is then given to the project’s procurement team so they can buy all the cables on the list, then to the electricians, so that they know how many cables they have to install, and so on.

This data-transformation work - requirements and manufacturer’s data in, design calculations out - is very repetitive, but absolutely necessary for a successful engineering design. It is important that the correct data is put in; that the transformations are applied correctly; and the data is output in a way that makes it easy for the intended audience to use.

The second dirty secret of engineering is that a shockingly large amount of this repetitive busy-work is still done manually.

An example: Most industrial projects involve electric motors. The mechanical engineer maintains a spreadsheet of motors with mechanical data (torque, power, etc.) The electrical engineer keeps a separate spreadsheet with electrical data (kilowatts, amps, power factor, etc.) The two spreadsheets are synchronised by printing paper copies of each spreadsheet and manually comparing them line by line.

This process takes hours and is an amplifier, nay, a factory for human errors.

A typical story is that an error is overlooked while synchronising the two lists, and the wrong motor is ordered, manufactured, and delivered to site. The error is only discovered when the installation crew comes to install the motor and finds that it doesn’t fit, it’s the wrong size, and doesn’t couple onto the machine.

I am not sure why processes like this are not automated. Computers are really good at comparing things and telling you about the differences between them. It can be done with Excel formulas, or VBA, or (possibly the best choice) Unix’s diff utility.

Why isn’t this done?

• Tradition: “We’ve always done it manually.”
• Barriers to entry: “I don’t have the time to learn how to do that.”
• Lack of awareness: “I didn’t even know there was a tool for that.”
• Wanting a human involved in the process: “There are some special cases that the computer wouldn’t know how to handle.”

None of these obstacles are insurmountable.

• “We’ve always done it manually.” → Tradition is the hobgoblin of small minds. If a computer can do it faster and with less errors, it should be done by a computer. We aren’t paid to sweat and slave over a design; we’re paid to deliver results.

• “I don’t have the time to learn how to do that.”

  → There is a great opportunity here for engineers who know a programming language: you can make a living by automating the boring parts of the engineering task, giving your team more time for the interesting parts. The amount of time you save will more than justify your salary.

• “I didn’t even know there was a tool for that.”

  → Engineers who are familiar with the tools used by programmers and IT people can draw on that knowledge to suggest powerful tools that help with the engineering task.

• “There are some special cases that the computer wouldn’t know how to handle.”

  → An opportunity for engineers with technical understanding of the special cases to write a program that correctly handles the special cases.

• “The design process can’t be automated. It always needs a human to interpret the data and apply their engineering judgement.”

  → Absolutely true; there is no substitute (yet) for human experience, especially experience of design failures. This is why it is vitally important to automate as much of the busy-work as possible, so that the engineer has more time to apply their technical skills.

  IBM Pollyanna Principle

  (axiom) “machines should work; people should think.”

  • e.g. machines should do all the hard work, freeing people to think (hence the reference to Pollyanna)
  • e.g. most of the world’s major problems result from machines that fail to work, and people who fail to think.

All of which is leading to the main point of this article:

• Engineering is a process of input data → data transformation and calculation → output design. This is especially true for detailed design, but also holds true for concept design - the input data is very nebulous, and the output design is very high-level, but it is still input → processing → output.

• Computers are very good at data transformation and calculation.

  Computers can’t make nuanced judgements, but that doesn’t matter.

  If engineering is ten parts hard work to one part inspired thinking, and a computer can take half of the hard work off your shoulders, you now have twice as much time for inspired thinking.

• Therefore, an engineer who knows about computer programming and related information-technology will be very efficient at turning data into designs.

This is why I recommend that all engineers know how to program in at least one language; or, at the very least, be aware of the savings they can realise by having a programmer on their team.

The bright side is that if you are a technically knowledgeable engineer, who also knows how to program computers, you will have no problems finding employment. Once you demonstrate how much you can improve your team’s productivity, you will have a constant stream of work - some of it very interesting and all of it worth while.

As Zed Shaw put it (emphasis mine):

Which programming language you learn and use doesn’t matter. Do not get sucked into the religion surrounding programming languages as that will only blind you to their true purpose of being your tool for doing interesting things…

Programming as a profession is only moderately interesting. It can be a good job, but you could make about the same money and be happier running a fast food joint. You’re much better off using code as your secret weapon in another profession.

People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines.

Thanks to Billy Rittson, who critiqued the draft of this article.