Skip to content

Instantly share code, notes, and snippets.

@adamcameron
Created January 15, 2013 08:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save adamcameron/4537246 to your computer and use it in GitHub Desktop.
Save adamcameron/4537246 to your computer and use it in GitHub Desktop.
Some code to look at how much accuracy a double has
<cfscript>
struct function compareStringToNumeric(string s){
var result = {
asString = s,
asNumber = val(s)
};
result.equal = !compare(result.asString, result.asNumber);
return result;
}
integerOnly = "";
decimalOnly = "";
withBoth = "";
intPart = "";
decPart = "";
xls = spreadsheetNew();
</cfscript>
<table border="1">
<tr>
<th rowspan="2">Digits</th>
<th colspan="3">Integer</th><th colspan="3">Decimal</th><th colspan="3">Both</th>
</tr>
<tr>
<th>String</th><th>Number</th><th>Equal</th>
<th>String</th><th>Number</th><th>Equal</th>
<th>String</th><th>Number</th><th>Equal</th>
</tr>
<cfloop index="i" from="1" to="20">
<cfif i mod 10> <!---omit zeros as they will be ignored if trailing--->
<cfscript>
integerOnly &= (i mod 10);
decimalOnly = "0.#integerOnly#";
if (i mod 2){
intPart &= (i mod 10);
}else{
decPart &= (i mod 10);
}
withBoth ="#len(intPart)?intPart:0#.#len(decPart)?decPart:0#"; // make sure the decimal point is handled the same as the numeric will be (leading & trailing zero if necessary)
intCompared = compareStringToNumeric(integerOnly);
decCompared = compareStringToNumeric(decimalOnly);
withBothCompared = compareStringToNumeric(withBoth);
</cfscript>
<cfoutput>
<tr>
<td>#i#</td>
<td>#integerOnly#</td>
<td>#intCompared.asNumber#</td>
<td>#intCompared.equal#</td>
<td>#decimalOnly#</td>
<td>#decCompared.asNumber#</td>
<td>#decCompared.equal#</td>
<td>#withBoth#</td>
<td>#withBothCompared.asNumber#</td>
<td>#withBothCompared.equal#</td>
</tr>
</cfoutput>
<cfscript>
spreadsheetSetCellValue(xls, i, i, 1);
spreadsheetSetCellValue(xls, integerOnly, i, 2);
spreadsheetSetCellValue(xls, intCompared.asNumber, i, 3);
spreadsheetSetCellFormula(xls, "if(B#i#=C#i#,true, false)", i, 4);
spreadsheetSetCellValue(xls, decimalOnly, i, 5);
spreadsheetSetCellValue(xls, decCompared.asNumber, i, 6);
spreadsheetSetCellFormula(xls, "if(E#i#=F#i#,true, false)", i, 4);
spreadsheetSetCellValue(xls, withBoth, i, 8);
spreadsheetSetCellValue(xls, withBothCompared.asNumber, i, 9);
spreadsheetSetCellFormula(xls, "if(H#i#=I#i#,true, false)", i, 4);
</cfscript>
</cfif>
</cfloop>
</table>
<cfset spreadsheetWrite(xls, expandPath("./") & "result.xls", true)>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment