4.1 Decimal Numbers
D.M. Donner
Machinists live in a world ruled by decimals. This is true whether we are using the metric or inch units of measure. Whole numbers are relatively easy as they indicate a quantity of complete, rather than partial, units. When we manufacture parts with precision tolerances, we must practice principles that support accuracy. Precision is a term you will hear quite a bit in the manufacturing industry, and it means being exact or accurate. Accuracy is how close a measured feature is to the intended dimension. You are entering into a trade which has its own language and that is based on numbers and accuracy. Read this chapter as often as necessary to master this language.
Decimal Theory
Decimal place refers to how many digits after the decimal the machinist needs to take into consideration when measuring a dimension. These digits are directly related to the level of resolution required by the design engineer. Resolution is the smallest increment of measurement that can be identified. In general, the more decimal places a number has, the finer the resolution, and the higher tolerance translates to less deviation from the stated value.
When using decimal places to express a value, each digit to the right of the decimal point represents a fraction of a whole unit based on powers of 10. For example:
One decimal place: .1 represents one-tenth (1/10) of a unit. There are ten of these .1’s in an inch. .1″ is a little smaller than an 1/8th of an inch or .125″
Two decimal places: .01 represents one-hundredth (1/100) of a unit. There are 100 of these .01’s in an inch. This is about 3 sheets of copier paper.
Three decimal places: .001 represents one-thousandth (1/1000) of a unit. There are 1000 of these .001’s in an inch. A thousandth is about the size of a human hair.
Four decimal places: .0001 represents one ten-thousandth 1/10,000) of a unit. This resolution is rather challenging to grasp because it is hard to identify objects to demonstrate it. We need powerful magnification to see things this small. Because heat can cause a part to grow by ten-thousandths (.0001″), micrometers have plastic covers on them to isolate the heat from your hand, which would affect the dimension. A micrometer is a gage that reads small distances.
Talk Like a Machinist
Our numerical system is based on the power of ten. Because of this, decimal places represent divisions by tens, and the first decimal place is the tenth’s position and is read as tenths of an inch. This convention works just fine for many occupations, such as weather reporting on the local news. If it rained .1″ last night, they would report “a tenth” of an inch of rain fell last night. Weather reporting does not require the same degree of precision that machining does. In fact, it may just be the only job where you can be wrong 50% of the time (National Oceanic and Atmospheric Administration, 2024) and still keep your job. Machinists need to be more exact in their measurements and this is where decimals come in.
Most of the time, machinists work with .001″ (one thou) tolerances. The practice of working with such high tolerance has created a machinist vocabulary when referring to decimal numbers. The language developed as the base 1000 system, in which all numbers are expressed as if they were in thousands or three decimal places, i.e., .001″.
How a machinist reads these inch numbers is as follows:
.1= one hundred thousandths (one hundred thou)
.2= two hundred thou
.01= ten thousandths (ten thou)
.03= thirty thou
.001= one thousandths
.125= a hundred and twenty five thousandths
.0001= one ten-thousandths (one tenth)
.3751= three hundred and seventy-five thou and one tenth
.3755= three hundred and seventy-five thou and five tenths
Notice we save the term “tenths” for four decimal places, where traditionally it would be used for the first decimal place. Tenths are outside the scope of this book and are not likely encountered by entry level machinists, so we will stick to the thousandths as the highest resolution for our projects.
the complete quantity of units we are measuring such as an inch or a millimeter
the process of being exact or accurate
how close a measured feature is to the intended dimension
how many digits after the decimal the machinist needs to take into consideration when measuring a dimension
the smallest increment of measurement in a numerical value or measurement
a gage that measures small distances