All would be well if we could be sure that the user jrun (i.e. the user that owns our ColdFusion process) was indeed the only user ever to access this directory structure. In our case we want to be able to access this structure with PHP, too, which runs as mod_php on the webserver, thus as user www-data. To avoid permission problems, we want to assign a permission of 0777 to all directories in the structure upon creation.

You'll need at least JDK 1.6 for this to work:

<cffunction name="createWRXPath" 
	access="public" 
	returntype="boolean" 
	output="no">
 
	<cfargument name="strPath" 
		type="string" 
		required="yes" 
		hint="an absolute physical path 
		to a directory, not a file">
 
	<cfargument name="iMinDepth" 
		type="numeric" 
		required="no" 
		default="4" 
		hint="minimum depth to set permissions">
 
	<cfscript>
 
		var local = structNew();
 
		local.strPath = GetDirectoryFromPath(
			JavaCast('string',arguments.strPath));
 
		local.objFile = CreateObject('java'
			,'java.io.File').init(local.strPath);
 
		if (arguments.iMinDepth gt 4) {
			local.iMinDepth = arguments.iMinDepth;
			} else {
			local.iMinDepth = 4;			
		} // end if (arguments.iMinDepth gt 4)
 
		// create the path
		local.objFile.mkdirs();
 
		// if that hasn't succeeded, return false
		if (not DirectoryExists(local.strPath)) return false;
 
		local.l = ListLen(local.strPath,'/');
 
		local.strPartPath = '/';
 
		// now descend into the path from top down, 
		//   set permissions for every directory lower 
		//  than level iMinDepth
 
		for (local.i=1; local.i lte local.l; local.i++) {
 
			local.strPartPath &= 
				ListGetAt(local.strPath,local.i,'/') & '/';
 
			if (local.i gte local.iMinDepth) {				
				local.objFile = CreateObject('java'
					,'java.io.File').init(
					    local.strPartPath);
 
				local.objFile.setReadable(TRUE,FALSE);				
				local.objFile.setWritable(TRUE,FALSE);
				local.objFile.setExecutable(TRUE,FALSE);
 
			} // end if (local.i gte local.iMinDepth)
 
		} // end for (local.i=1; local.i lte local.l; local.i++)
 
		return TRUE;
 
	</cfscript>
 
</cffunction>

Example for usage:

<cfscript>	
	variables.strFile  = '/var/www/MYCACHE/123/123456/123456789.cache';
	variables.strPath  = GetDirectoryFromPath(variables.strFile);
	variables.bSuccess = createWRXPath(variables.strPath,4);	
	writeOutput(variables.bSuccess);
</cfscript>

So to get a date from a Unix timestamp in ColdFusion, you can use this oneliner:

<cfset dtMyDate = createObject('java','java.util.Date').init(javaCast('long',iUnixTS*1000)) />

ColdFusion string literals are just plain old Java strings. If you grab a string from e.g. a query object like variables.qMyQuery.myTextColumn, you need to be careful though - even if you think you just have one tuple returned, you've got something other than a string object on your hands. In such a case you need to either specifically target a certain row (like variables.qMyQuery.myTextColumn[1]) or you wrap it up in a JavaCast like Javacast('string',variables.qMyQuery.myTextColumn).

I finally found a moment to actually do some benchmarking on some of the built-in ColdFusion functions against their Java counterparts. This is not a benchmark of Java vs. ColdFusion performance, mind you, it's about deciding whether to use Java-methods inside of ColdFusion vs. ColdFusion's built-in string functions.

The test subject here is a string of 2848 characters length, pulled from our live article database, which should provide a realistic use case. The JavaCast has already been done, so if we're working with string data that may not be represented by actual Java string objects at the time of access, we may need to account for some additional processing cycles to do the JavaCast in order to be able to use Java methods on it. I ran the tests several times on the same machine, making sure the results where actually reproducable on each run. The tests have been performed on ColdFusion 8 Enterprise running in a 64-bit multi-instance configuration. If you're running CF9, your mileage may vary, though I wouldn't expect the results to point in a very much different direction.

On to the tests.

len() vs. length()

<cftimer type="outline" label="coldfusion len">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.iCfLen = len(variables.strText) />
	</cfloop>
</cftimer>
<cfoutput>#variables.iCfLen#</cfoutput>
 
<cftimer type="outline" label="java length">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.iJLength = variables.strText.length() />
	</cfloop>
</cftimer>
<cfoutput>#variables.iJLength#</cfoutput>

Results: ColdFusion 76ms vs. Java 104ms.

Find() vs. indexOf()

<cftimer type="outline" label="coldfusion find">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.iCfFind = Find('Singularity',variables.strText) />
	</cfloop>
</cftimer>
<cfoutput>#variables.iCfFind#</cfoutput>
 
<cftimer type="outline" label="java indexOf">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.iJindexOf = variables.strText.indexOf('Singularity') />
	</cfloop>
</cftimer>
<cfoutput>#variables.iJindexOf+1#</cfoutput>

Results: ColdFusion 67ms vs. Java 115ms.

& vs. concat()

<cftimer type="outline" label="coldfusion &">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strCFconcat =  variables.strText & variables.strText/>
	</cfloop>
</cftimer>
 
<cftimer type="outline" label="java concat()">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strJconcat = variables.strText.concat(variables.strText) />
	</cfloop>
</cftimer>

Results: ColdFusion 120ms vs. Java 172ms.

ReReplace() vs. replaceFirst()

<cftimer type="outline" label="coldfusion ReReplace">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strCfReReplace = ReReplace(variables.strText,'Singularity','Singularity 2', 'ONE') />
	</cfloop>
</cftimer>
 
<cftimer type="outline" label="java replaceFirst">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strJreplaceFirst = variables.strText.replaceFirst('Singularity','Singularity 2') />
	</cfloop>
</cftimer>

Results: ColdFusion 331ms vs. Java 228ms.

ReReplace() vs. replaceAll()

<cftimer type="outline" label="coldfusion ReReplace">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strCfReReplace = ReReplace(variables.strText,'Singularity','Singularity 2', 'ALL') />
	</cfloop>
</cftimer>
 
<cftimer type="outline" label="java replaceAll">
	<cfloop from="1" to="10000" index="variables.i">
		<cfset variables.strJreplaceFirst = variables.strText.replaceAll('Singularity','Singularity 2') />
	</cfloop>
</cftimer>

Results: ColdFusion 543ms vs. Java 629ms.

Conclusion

There seems to be a certain overhead involved in using the underlying Java methods instead of the build-in ColdFusion functions (or operators in case of the &). The difference is actually not that much, and depending on the data you work on, the Java approach might actually gain the upper hand in some cases.

However, based on these findings I would suggest you stick with ColdFusion for such simple cases; your CFML code will be more independent of the underlying VM (which may even be .NET) and easier to write, too, as most ColdFusion IDEs will not support mixed code completion for both CFML and Java.

This is no reason not to take a look behind the curtain, though. Where Java really shines from a ColdFusion developers view is in solving problems that couldn't be as elegantly solved in plain CFML. For example, you'll be able to implement a buffered stream writer instead of <cffile action="append" />, you'll have java.util.Calendar to do your bidding working with date/time-data, and you'll be able to use a wide range of external Java libraries from within ColdFusion, solving both trivial and most complex problems without you doing all the legwork coding the solutions.

On the other hand, when you actually have discovered some of the gems that are hidden in the Java foundation of ColdFusion, don't just blindly adopt everything you find there. You still need to do some testing to find the tool that fits your needs best.

<cffunction name="CF2UnixTS" access="public" output="no" returntype="numeric"
	hint="calculates a Unix Timestamp from a DateTime value">
	<cfargument name="tsTheDate" type="date" required="yes">
	<cfscript>
	var local=structNew();
	local.objCalendar = createObject('java'
				,'java.util.Calendar').getInstance();
	local.objCalendar.clear();
	local.objCalendar.set(year(arguments.tsTheDate)
			,month(arguments.tsTheDate)-1
			,day(arguments.tsTheDate)
			,hour(arguments.tsTheDate)
			,minute(arguments.tsTheDate)
			,second(arguments.tsTheDate));
	local.iUnixTS = int(local.objCalendar.getTimeInMillis()/1000);
	return local.iUnixTS;	
	</cfscript>
</cffunction>

Usage example:

<cfscript>
	dtLocal = DateAdd('M',7,now());
	iUnixTS = CF2UnixTS(dtLocal);
	iDateDiff = DateDiff('s',CreateDate(1970,1,1),dtLocal);
</cfscript>
<cfquery name="getTS" datasource="#APPLICATION.strSomeDS#">
	SELECT UNIX_TIMESTAMP(<cfqueryparam cfsqltype="cf_sql_varchar" 
			value="#DateFormat(dtLocal, 'yyyy-MM-dd' ) 
			& ' ' & TimeFormat(dtLocal, 'HH:mm:ss' )#">
			) AS uxts;
</cfquery>
<cfoutput>
	dtLocal : #dtLocal# <br />
	iUnixTS : #iUnixTS# <br />
	getTS.uxts: #getTS.uxts# <br />
	DateDiff: #iDateDiff#
</cfoutput>

This gives me

dtLocal : {ts '2010-12-28 15:33:14'}
iUnixTS : 1293546794
getTS.uxts: 1293546794
DateDiff: 1293550394

The values from the UDF and the one generated by MySQL are the same. The DateDiff-value is off by minus one hour exactly.

Maybe I am doing something overly complicated here. Is there some easier way to get at the Unix timestamp of a date in ColdFusion?

Verity never really quite met all of our needs and was a real pain to set up and maintain. CF 9's Solr, which is based on Lucene, might be a mighty step forward, but we're still running on ColdFusion 8, so I really cannot say a lot about handling and performance of the new indexing beast.

For our use cases (i.e. indexing of articles, products in our CMS as well as our forums), Sphinx (for SQL Phrase Index) has shown some amazing results - and we're using it for a couple of months now. In this article I'll show you how to compile, set up and use Sphinx in your ColdFusion application to retrieve search results from documents stored in a PostgreSQL or MySQL database.

At the time of writing this article, the current version of Sphinx is 0.9.9-release. I assume you intend to deploy the Sphinx server on a Linux box (I'm using Debian Lenny) and are familiar with compiling code - if you're on Windows, you should get the binary distribution and see docs for installation.

The server you'll want to run Sphinx may or may not be the same as your ColdFusion box, though I'd recommend you set up a separate machine with a generous amount of RAM - Sphinx will store indexes completely in memory which explains the amazing search speed, so the more data you wish to index, the more RAM you should provide.

Next page: Compiling Sphinx

Of course I could use a nested loop to do this - but for longer lists this is neither fast nor elegant. Again I'll turn to Java for this. ColdFusion's arrays are in fact java.util.Lists, so after converting our ColdFusion lists to ColdFusion arrays, we can make use of the Java-API for lists.

Here's a quick UDF that does what I want:

<cffunction name="listIntersect" output="no" returntype="string" 
	hint="returns values from list 1 which are contained in list 2">
	<cfargument name="lstSand" type="string" required="yes" />
	<cfargument name="lstSieve" type="string" required="yes" />
	<cfargument name="chDelimiter" type="string" required="no"
		    default="," />
	<cfscript>
	var aLstSand  = listToArray(arguments.lstSand,arguments.chDelimiter);
	var aLstSieve = listToArray(arguments.lstSieve,arguments.chDelimiter);
	aLstSand.retainAll(aLstSieve);
	return arrayToList(aLstSand,arguments.chDelimiter);
	</cfscript>
</cffunction>

Usage:

<cfset lstSand   = 'foo,bar,illegalparam,whatever' />
<cfset lstSieve  = 'bar,foo,someotherval,whatever' />
<cfset lstSieved = listIntersect(lstSand,lstSieve) />
<cfoutput>#lstSieved#</cfoutput>

This will output foo,bar,whatever.

The Unix timestamp of 2038-01-19 03:14:07 is 2147483647. This is the maximum number that fits into the int4 data type. One second later we'll be getting integer overflow for any operations on Unix timestamps. Like getting the actual date from that Unix timestamp via dateAdd() in ColdFusion.

In our case, some user had entered a date somewhere in the 2040ies into our PostgreSQL database - which is working fine, as PostgreSQL actually has not only some sophisticated timestamp datatypes but also a couple of handy functions and operators to go with it.

For searching however we're using Sphinx, a very nice full text search engine. The results returned by Sphinx will use Unix Timestamps for date values - so here we get hit by the next Y2K problem. To convert such Unix Timestamps in ColdFusion, we'd usually do something like this:

<cfset dtMyDate = dateAdd("s", iUnixTS, "01/01/1970") />

If iUnixTS has a value of more than the maximum allowed value for int4, which is 2147483647, you get an exception like

Cannot convert the value 2.147607103E9 to an integer because it cannot fit inside an integer.

You'll have a hard time dealing with int8-values in ColdFusion. Java to the rescue:

<cffunction name="unixTStoDate" access="private" output="no" returntype="date">
<cfargument name="iUnixTs" required="yes" type="numeric">
<cfscript>
var iMaxLongInt = 2147483647;
var objBigint = createObject('java', 'java.math.BigInteger').init(1);
var longUnixTs = objBigint.valueOf(javaCast('long',arguments.iUnixTs));
var longMaxLongInt = objBigint.valueOf(javaCast('long',iMaxLongInt));
var iUnixTsLongIntFactor = int(arguments.iUnixTs / iMaxLongInt);
var iUnixTsLongIntMod = longUnixTs.mod(longMaxLongInt);
var i=0;
var tsReturn = '01/01/1970';
// we'll just DateAdd the maximum allowed value for
// as often as needed...
for (i=1; i lte iUnixTsLongIntFactor; i++) {
tsReturn = DateAdd('s',iMaxLongInt,tsReturn);
} // end for (i=1; i lte iUnixTsLongIntFactor; i++)
// ... and then dateAdd the rest
tsReturn = DateAdd('s',iUnixTsLongIntMod,tsReturn);
return tsReturn;
</cfscript>
</cffunction>

Let's try this:

<cfset longUnixTS = 2147483647 + 123456 />
<cfset dtMyDate = dateAdd("s", longUnixTS, "01/01/1970") />
<cfdump var="#unixTStoDate(longUnixTS)#" />

Result:

{ts '2038-01-20 13:31:43'} .

Hooray!

As I started writing the search component for our articles, I decided that it would be an extremely valuable feature to use the weighting functionality Sphinx provides. This would allow to set different weights to the indexed fields on a per-query basis, which is extremely cool as you don't have to decide on a general per-field weighting setting during indexing.

The Sphinx API documentation does mention a SetFieldWeights-method which takes a HashMap (i.e. an associative array for those of the PHP persuasion) as argument. I constructed the Hashmap like this (excerpt from a function):

var jHashMap = createobject("java", "java.util.HashMap");
jHashMap.put('keywords',JavaCast('int',arguments.iWeightType));
jHashMap.put('text',JavaCast('int',arguments.iWeightText));

So far, so good. Now I tried calling

variables.sphinx.SetFieldWeights(jHashMap);

- and received an error that this function does not exist. One dump of the Sphinx-API later, I saw the error was a mere typo: In org.sphx.api.SphinxClient the method was actually spelled SetFieldeights. Being a responsible user, I did report this bug in the forums, but what to do for the time being? Two options: Correct the spelling mistake in the API-source, recompile and redeploy - and risk that a subsequent update might have to be patched as well (and one tends to forget something like that) or just accept the typo in my own code. I sort of opted for the latter, but I wanted to be equally safe from a later correction of this typo in an updated API.

So I wanted to make my code work with both SetFieldeights or SetFieldWeights. This could be done either by CFTRY/CFCATCHING an error on calling the method (something I try to avoid at all costs), or simply checking if the method is implemented by the class.

Now my first try with structKeyExists() failed. Apparently I cannot test a java object as if it were a structure, or probably more like I cannot test a method of such an object as if it were a member of a conventional structure. isDefined() failed, probably for similar reasons. Okay, on to the good Java stuff. Java provides reflection for such cases, in fact this is what ColdFusion uses when you cfdump such an object.

To make this reusable, decided to write this as as more generic function I could use in another CFC:

<cffunction name="methodExists" returntype="boolean" access="public" output="no" hint="checks if a java class implements a method">
	<cfargument name="strClassName" type="string" required="yes">
	<cfargument name="strMethodName" type="string" required="yes">
	<cfscript>
		var a = arrayNew(1);
		var myClass = a.getClass().forName(arguments.strClassName);
		var arrMethods = myClass.getDeclaredMethods();
		var i = 1;
		var arrMethodNames = arrayNew(1);
		for (i=1; i lte ArrayLen(arrMethods); i++) {
			arrMethodNames[i] = arrMethods[i].getName();
		}
		if (arrMethodNames.indexOf(arguments.strMethodName) eq -1) {
			return false;
			} else {
			return true;
		}			
	</cfscript>
</cffunction>

Now to check if the Sphinx-API implements SetFieldWeights, I just need to test the boolean return value like

<cfscript>
if (methodExists(strClassName='org.sphx.api.SphinxClient',strMethodName='SetFieldWeights')) {
  variables.sphinx.SetFieldWeights(jHashMap);
  } else {
  variables.sphinx.SetFieldeights(jHashMap);
}
</cfscript>

Mission accomplished! For those curious to know, the affected Sphinx version was sphinx-0.9.9-rc2.

If all you want to do is allow your users to store some plain old text, maybe seasoned with some kind of BB-code markup to allow for some limited text formatting, this method is just fine. If that is not enough and you actually need to allow a limited amount of good old HTML, you'll need some more sophisticated sanitizing mechanism to parse out any potentially harmful code elements like JavaScript actions and the like.

Users' HTML input may be harmful in more than one way - if it is not well formed, it may mess up the rest of your page; take a look at my earlier post on this issue "Coldfusion UDF-wrapper for JTidy to clean up HTML". In this post I promised to also write about the other evil that lurks in user input and how to fight it if HTMLEditFormat() is doing too much collateral damage in your use case.

A little while ago, a guy named Samy Kamkar rocked the blogosphere with his spectacular yet simple XSS-worm which gathered myriads of "friends" for him, because Samy was their hero. In honour of Samy, the Open Web Application Security Project (OWASP) gave birth to an extremely useful API to combat Samy copycats, along with implementations for .NET and - hooray! - Java, named the offspring AntiSamy, and generously put everything under a BSD license.

AntiSamy is not just a simple RegEx-Replacer that silently drops a certain set of potentially malicious code elements. You can define your own ruleset inside a policy file - or rulesets (i.e. policy files) for that matter - maybe you've got some group of privileged users whose input should be treated not quite as strict as that of your average Joe WannabeH4XX0r. And AntiSamy won't just do the sanitizing, it will also allow you to tell the user what has been rejected and why, as there's quite some potential for frustration if he designs some beautifully CSSed code and your policy chucks out all the good stuff.

Speaking of good stuff, let's get onto it.

First you'll need to head to the AntiSamy download page on Google Code. Grab the latest antisamy-bin.jar (antisamy-bin.1.3.jar at the time of this post) and drop it in your ColdFusion class path (e.g. /opt/coldfusion/lib in a standalone-server install). Restart ColdFusion.

On the download page you'll also find a couple of pre-made policy files. The slashdot policy file is the most resttrictive one, followed by the ebay- and myspace- files. The anythinggoes-file is too permissive for any serious application, so you may as well ignore that one. Put those policy files in a new AntiSamy-policy folder somewhere on your server - the files need not be web-accessible, so you may well use something like /var/local/lib/AntiSamyPolicies/. You may of course build your own policy file based on one of the existing examples and drop it in that same folder.

Now to use this from ColdFusion, you may grab my sanitizer.cfc:

<cfcomponent displayname="sanitizer" hint="sanitize user generated HTML from malicious elements">
        <!--- call init() automatically when the CFC is instantiated --->
        <cfset init()>
 
        <cffunction name="init" returntype="sanitizer" access="public" output="false">
                <cfargument name="policy" default="myspace" type="string" required="no">
                <cfscript>
                        variables.policyVersion = '1.1.1';
                        variables.listPolicyDefaults = 'anythinggoes,ebay,slashdot,myspace';
                        variables.defaultPolicy = 'myspace';
                        variables.policyFilePath = '/var/local/lib/AntiSamyPolicies/';
                        variables.policyFile = variables.policyFilePath & setPolicy(arguments.policy);
                        this.sanitizer = createObject("java", "org.owasp.validator.html.AntiSamy");
                        return this;
                </cfscript>
        </cffunction>
 
        <cffunction name="setPolicy" returntype="string" access="private" output="false">
                <cfargument name="sPolicy" required="yes" type="string">
                <cfscript>
                        var sFullPolicyPath = variables.policyFilePath & 'antisamy-';
                        arguments.sPolicy = lCase(arguments.sPolicy);
                        if (ListFind(variables.listPolicyDefaults,arguments.sPolicy,',')) {
                                return 'antisamy-' & arguments.sPolicy & '-' & variables.policyVersion & '.xml';
                        } else {
                                // in this case we might need to load a custom policy file
                                sFullPolicyPath = sFullPolicyPath & arguments.sPolicy & '-' & variables.policyVersion & '.xml';
                                if (FileExists(sFullPolicyPath)) {
                                        return 'antisamy-' & arguments.sPolicy & '-' & variables.policyVersion & '.xml';
                                } else {
                                        return 'antisamy-' & variables.defaultPolicy & '-' & variables.policyVersion & '.xml';
                                }
                        }
                </cfscript>
        </cffunction>
 
        <cffunction name="sanitize" access="public" output="false" returntype="s
                <cfargument name="sTaintedHTML" default="" required="yes" type="
                <cfargument name="sPolicy" default="" required="no" type="string
                <cfscript>
                        var objResults = 0;
                        if (arguments.sPolicy neq '') {variables.policyFile = va
                        objResults = this.sanitizer.scan(arguments.sTaintedHTML,
                        this.arrayErrorMessages = objResults.getErrorMessages();
                        return objResults.getCleanHTML();
                </cfscript>
        </cffunction>
 
</cfcomponent>

To use this component, you would instantiate an AntiSamy instance for each required policy in your request scope for each request where you need this component:

<cfscript>
	request.objSanitizer=CreateObject('component','antisamy.sanitizer').init('slashdot');
</cfscript>

Now you can process your user's input:

<cfscript>
	variables.sSanitizedHTML=request.objSanitizer.sanitize(variables.sSomeTaintedHTML);
	if (arrayLen(request.objSanitizer.arrayErrorMessages) gt 0) {
		// do some error handling like displaying the errors to the user
	} // end if (arrayLen(request.objSanitizer.arrayErrorMessages) gt 0)
</cfscript>

You may now store and output the processed HTML and also display information about what sanitizing changed in the original HTML to the user - these messages are available in the request.objSanitizer.arrayErrorMessages-array.

I hope some of you might find this useful!

BTW - for those who need functionality like this in PHP, maybe HTML Purifier will suit your needs.