Silent Setup of Sun JDK (Linux, Debian)

I was looking for a way to install the latest Java Runtime Environment without any user interaction and came across this posting

Here's a tips I found trying to do the same as you :
1. Create an answers.txt file with the word yes inside
2. Launch fhe following command (assuming you want to install jdk-xx.bin)
./jdk-xx.bin <>/dev/null

Works like a charm!

Ant & SCP on Mac

Just spend a few minutes involuntarily on fixing the SCP support for ANT on Mac ... a library is missing ("Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.Scp was not found.") and you have to copy a second file from the binary distribution, check out the howto I found on the web!

Important hint: Don't download the latest binary distribution, check which one you're running (ant -version) before doing that! Otherwise you'll likely receive error messages again.

NYCFUG - I was there ;-)

Some nice memories of my stay in NYC this summer - a picture with the famous CF blogger Ben Nadel at the NY CFUG! Two thumbs up!

Amazon EC2 goes Lorem ipsum

"Lorem ipsum" is the typical dummy text so I was quite surprised to read it on the Amazon EC2 Console (Setup wizard for a new instance) just a few seconds ago (see screenshot below). I guess I was just using the interface during an upgrade ... at least it reminds me a little bit of the pain during my Latin lessons in school ... ;-)

Add missing Array.indexOf to Internet Explorer

When dealing with JSON it comes in handy to get the position of a certain string in an array, image an array with column names returned by a SerializeJSON'ed CFQUERY.

Internet Explorer does not support this function, but just add the following snippet and you can use Array.indexOf( 'TITLE' ) in any browser without built-in support as well (We add a new function to the Array prototype!)

// make sure indexOf is supported
if(!Array.indexOf){
   Array.prototype.indexOf = function(obj, start){
       for(var i=(start||0); i<this.length; i++){
           if(this[i]==obj){
               return i;
           }
       }
   }
}

Apache, Railo & mySQL Troubles after Upgrading to Snow Leopard

Odd. The upgrade to snow leopard just killed by Apache Resin Connector and the vhost config - that should not happen (read other reports here). So make sure you backup your vhost configs before upgrading. For the connector, I had to re-download XCode in order to build the connector ... (700 MB in total, take care!).

The mySQL server wasn't starting as well, a symbolic link seems to be broken after the update, see here. Furthermore, some programs like Cyberduck (SSH) and Nambu (twitter) aren't working any more. To be honest, a quite disapointing update so far.

Setup Mac OS X + Apache + Railo

A fews days ago I upgraded to a Macbook Pro and I already spent almost two days on reinstalling software, copying documents etc etc (An upgrade is an opportunity for me to get rid of old software and ballast).

Today I started with the Apache / railo setup using a guide by Luid Majano (I had to download the Apple XCode Setup file (almost 1 GB) in order to compile the connector for Apache and reset the httpd to 32 bit instead of 64 bit).

Further problems I had to solve:

• Read how to fix the "Apache Error: Client denied by server configuration" error (very restrictive configuration makes sense on real life systems but on dev systems ...?)

• I wasn't able to access my files until I executed a chmod a+x on every directory in the path to the webroot (apache log [error] [client ::1] (13)Permission denied:)

Now it's working like a charm.

Using the H2 database in ColdFusion

Railo is delivered with an embedded database called H2. The description on the homepage says:

Welcome to H2, the Java SQL database. The main feature of H2 are:

* Very fast, open source, JDBC API

* Embedded and server modes; in-memory databases

* Browser based Console application

* Small footprint: around 1 MB jar file size

Such a database fit's perfectly for a current task at tunesBag - the streaming servers should store some data on their own machines and mysql or postgresql just would be overkill for this task. So I decided to give H2 a look and it looks pretty nice.

On my local machine I'm working with ColdFusion 8 as well - the configuration to add H2 support for CF is very easy, just follow the following steps:

• Download the latest version of H2
• Drop the h2*.jar file in the WEB-INF/lib directory of your ColdFusion server (directory depends on your type of setup)
• Restart CF
• Add a new datasource
• JDBC URL = jdbc:h2:file:%Path to Database file on your disk%
• Driver Class = org.h2.Driver
• Driver name = default
• User name = sa

That's it - the files holding the data will be created automatically for you. Take a look at the H2 tutorial in order to find out more (e.g. server mode etc).

Railo - Compatibility issues with scopes (URL etc)

I'm migrating my development environment to railo and found some issues that affected my code.

One of the most important ones is that railo always references the scope name when you use URL, FORM etc no matter if a local variable with this name exists or not. To give you an example, run this code in CF and in railo:

<cfparam name="url.step" type="string" default="1" />

<cfset a_call = PrintURL( 'http://tunesBag.com/' ) />

<cfdump var="#a_call#">

<cfscript>
function PrintURL( url ) {
return url;
}
</cfscript>

To make railo behave the same way, you've to add a full reference to the function -

return arguments.url

In ColdFusion you'll see http://tunesBag.com as output, in railo the URL structure. Some further explanations concerning the scopes and this "bug" can be found here (Adobe Livedocs) and here (Railo Bugtracker)

Having fun with railo: CFQUERY supporting complex objects

I just came across a cool feature of railo - it's possible to set query columns to complex objects, e.g. a java object or a structure.

An example:

cfset q = QueryNew('data');
querySetCell( q, 'data', { firstname = 'John', surname = 'Doe' }, 1);

cfdump var="#q#"

You can access the structure by using q['data'][1]!

cool!

Important: This complex data won't survive a query of queries operation, in this case the simple classname will be shown in the column.

CFFEED not usable due to missing timeout property

I've always used CFFEED in order to read newsfeeds (e.g. display recent blog postings on the homepage); now we had serious problems with non-responding feeds - and as it seems there is no workaround (see posting by Ben Nadel as well). As solution I decided to go back to cfhttp and simple parsing of the XML document.

"Corrupt table" errors (ColdFusion 8, mysql)

We have to do a lot of caching in order to keep the performance of tunesBag.com, so cached queries are very important for our website. During the last two days a lot of "corrupt tables" errors showed up in the log and first I thought about a mysql database issue of course. It's a bug in ColdFusion, however, so in case you run into the same error use this hotfix: http://kb.adobe.com/selfservice/viewContent.do?externalId=kb402583

I've updated our live servers and I hope we won't see this error again.

Enhance performance of Joins / Queries: Make sure all columns share the same collation

At tunesBag.com, we're storing most of the data in UTF-8 / Unicode because we've customers from all over the world. Some internal data like userkeys, entrykeys etc are stored in latin1 because it needs less space. I recently came accross a query which was very slow without any obvious reason and after some research I found out that in a left join, the column A from table A has a different collation than column B from table A.
I changed the collation for column B to the same as column A - and the query executive time dropped from about 1300 msec to 100 msec!

So always make sure you're using the same collation if possible when performing joins!

JavaScript variables set in onclick - different behaviour in IE / FF

I recently discovered that the behaviour of internet explorer and Firefox is quite different when it comes to settings variables in an onClick event - in Firefox these variables will become global variables, in IE they are just "local" and will nur be available for other functions.

An example:

a onClick="variableA = 'peter';CallFunction();return false;" ...

function CallFunction() {
alert(variableA);
}

In FF, an alert box will show "Peter", in IE you'll receive an exception.

How to solve that? Simply declare variableA on top of your JavaScript file, this way IE will modify the global variable.

var variableA = '';

Railo - important path placeholders

I'm playing around with railo, a full featured alternative CFML engine and so far, I'm very impressed. During testing my existing applications with this new environment I had to add some custom mappings in the administration (by the way, there are two admin engines, the first one is server-wide, the second one just for the current webroot).

When creating these custom mappings, there are some very interesting path placeholders you can use:
{railo-web}: path to the railo web directory typical "{web-root}/WEB-INF/railo"
{railo-server}: path to the railo server directory typical where the railo.jar is located
{temp-directory}: path to the temp directory of the current user of the system
{home-directory}: path to the home directory of the current user of the system
{web-root-directory}: path to the web root
{system-directory}: path to thesystem directory

This way you can create mappings without using the full path to the directory - very cool!

Create & Connect to databases (mysql, SQLite etc) on the fly with ColdFusion

Working with databases in ColdFusion is very easy - you have to set up your datasources in the CFAdmin (or by code using the CFAdmin API) and you can start writing SQL using CFQUERY. But what if you want to connect to a database on the fly without creating a datasource?
In this case it comes in handy that ColdFusion is a Java-based product, so you can use all the power JDBC is offering. At the online music hub tunesBag I recently had to create SQLite databases on the fly - it's quite easy in fact as you can see below:

1) Add the JDBC Driver (or use an existing one)
You can use any JDBC driver, e.g. mysql, MS-SQL etc
In my case: Download the SQLite JDBC Driver from zentus.com and place it in wwwroot/WEB-INF/lib (sqlitejdbc-v053.jar)

a_sqlite = createObject( 'java', 'org.sqlite.JDBC' )
a_prop = createObject( 'java', 'java.util.Properties' )
db_filename = '/tmp/sqlite_' & CreateUUID() & '.db'

2) Write the code (this is an example for sqlite)

a_conn = a_sqlite.connect( 'jdbc:sqlite:' & a_db_filename, a_prop.init() )
a_statement = a_conn.createStatement()
a_res = a_statement.execute( 'BEGIN;')

Create the table
a_res = a_statement.execute( 'CREATE TABLE test (id INTEGER PRIMARY KEY, firstname TEXT, surname TEXT);');

Insert data
prep = a_conn.prepareStatement( 'INSERT INTO test (firstname,surname) VALUES (?, ?);' );

prep.setString( 1, 'John' );
prep.setString( 2, 'Doe' );

// you can add multiple inserts ...
prep.addBatch();

prep.setString( 1, 'Max' );
prep.setString( 2, 'Mustermann' );
prep.addBatch();

// insert!
prep.executeBatch();

// save the changes
a_res = a_statement.executeUpdate( 'END;');

// quit
a_conn.close();
a_conn = 0;

Where to go from here
For more details, check out the official JDBC documentation provided by SUN!

mySQL - cfqueryparam - Comments - "No value specified for parameter"

I always try to add comments to complex queries against our mysql server and in order to see the comments in the debug output as well, I'm always using the /* comment syntax */ offered by SQL. The other day I added some comments to a rather complex query again and suddenly I received a strange error message all the time telling me "No value specified for parameter 4 / SQL State 07001".
After some playing around I found out that you should now use question marks in the SQL comments - it seems as CF or mysql is interpreting this char and assuming a parameter, even if it's contained in a comment.

New ColdFusion primer / tutorial

I've just finished a brand new tutorial for Adobe ColdFusion / CFML - in this primer I cover the basic server environment, database access, SQL, flow control, error handling and much more. Check it out here.

Not all CGI variables show up using CFDUMP

CGI variables must be something special - it is not possible to show all cgi variables when doing a cfdump. I just came across this bug when trying to use a custom #404 page where apache provides the original page URL in the CGI variable REDIRECT_URL.
I am just using CGI.REDIRECT_URL now and it works.
The interesting thing is that StructKeyExists(CGI, blabla) does not work at all - the check will always return true.

VARing cffile and the result variable

I am using cffile to upload incoming files and of course in my component I "var" the cffile variable at the top of the function (like one should do with cfhttp etc):

The problem is that somehow the cffile scope will stay zero - until I added the property "result" to the cffile tag saying the result should be stored as "cffile" - so the whole call is:



Now it works again!

Web Developer's Handbook

This endless list is like a Web Developers's Handbook at it's best ... a lot a page with stuff you already know but some interesting news as well

Apache: Disable HostNameLookups

After an upgrade to our infrastructure (see a picture here), the delivery of content through apache was incredible slowly - the reason was that we moved our DNS servers to an external location and had no internal server for this task any more.
After some research we found out that disabling the host name lookup (HostNameLookups = Off) helped to speed up the pages delivery. Now our analyzer will perform the lookup task and we're happy again ;-)

JRUN crashes after changing the hostname of a machine

Yeasterday we updated our system and renamed a server from www-4 to www04 in our internal DNS system. After that, this server started to deliver pages very slowly (with a very low load, however) and after some reasearch we found out that chaning the hostname was responsible.
So make sure your internal hostname and the hostname set in /etc/hostname is the very same!

MD5 Hash values: Be aware of uppercase/lowercase

These days I had to deal with a web service which needs a parameter as MD5 hash. I played around with for some time but authentification always failed, I did everything according to the documentation, however. Then I found out that the other service is checking the hash value case-sensitive, so I had to add a lcase to the string.
Some further research shows that almost every other service I know produces lowercase only hash values, ColdFusion produces uppercase values. Be aware of that! ;-)

JVM heap errors: Check every cffile / cfhttp request

This weekend I ran into trouble because the JVM (too small heap size) was crashing several times and after some research and debugging I found out that reading a 80 megabyte file with cffile was the reason for all the pain.
So be aware of this and do the following:

• Check the size of uploaded files (by performing checks with the size given in cffile or using a routine like that).
• Before you request any unknown website using cfhttp, do a HEADER operation and check out for the size (cfhttp.responseHeader, Content-Length) or add Content-Range headers to the request in order to allow only a certain content size
  

ColdFusion / MacOS 10.5 / WebConnector: Not again!!

Every update of apache is a real pain for me and maybe most of the ColdFusion community. Each time the server engine is updated, the WebConnector has to be updated as well. In the current case, I decided to update my Mac OS finally to Leopard (10.5) and - bingo - with the apache update from 1.3 to 2.2 the whole connector stuff isn't working any more.
One solution might have been again to download the whole connector sources (XCode) and compile the connector myself - some hours work plus about 1-2 GB downloads for an about 90kb file.
The other solution: Follow these instructions and download the file offered for download.

Attention: INNER JOIN is now the default in transfer

With the update to the latest release of the CF - based ORM system Transfer 0.6.3. , the default JOIN behaviour has been changed to INNER JOIN - I was wondering why some of my queries started to return a much smaller number of records at once ;-)
The solution was to write the full LEFT OUTER JOIN statement instead.

Very uncool - no implicit structure creation within function calls

I am using more and more features provided by CF8 as we migrate our servers to this new system. One feature I love is the implicit creation of structures without the need of calling StructNew() all the time.
So - a major downside is that you cannot create structures using this way within functions calls ... very bad. An example:

a_component.StoreInformation( key = arguments.key, data = { firstname= a_str_firstname } )

Very uncool, same thing noticed by Ben

Reset ColdFusion Administrator password

Today I wanted to edit a datasource name in the ColdFusion Administrator but I did not remember the password on this development server (hell, where are the nice little yellow post-its ;-)).
Here you can find a nice solution how to achive a password - reset.

XML vs JSON: Incredible performance differences

I admit - I am a real fan of XML and try to use this format whenever it is possible. The reason is simple: It's a clear format and very easy to exchange data with other applications (also with applications not running in a browser).
During the finalization of a project we came across several performance problems with a huge dataset and I tried to move the data interaction to JSON.
The result: The time of parsing a XML with jQuery (loop over the element using each) was up to 100 times higher than doing a simple JSON request (According to the results of the firebug console.timer output). No special treatment, just looping over the records and building an array in javaScript using the default jQuery way.

So, I hope in future times XML parsing will speed up with faster computers (although I think there will be not much gain in the near future because browser XML parsers just use MSXML or Xerces right now).
If you do not have ColdFusion 8 (which offers built-in JSON support), you might take a look at this library.

20 ways to secure your apache server

I can only recommend to go through the points of this posting if you want to secure your apache webserver.

Easy way to run several ColdFusion instances using one apache server

As you might know, in the server configuration JRun is the J2EE server in the background of CF and so in the apache configuration you will notice that the whole connector stuff is done using JRunConfig* directives.
As long as only one version of CF is running using apache as webserver everything is fine, it starts getting complicated when you decide to run e.g. 7 and 8 on one machine using one apache instance. Here you can find a solution how to set it up with different JRun ports which I did once as well (took me hours to find out all these stuff ;-)).
A very nice, quick and dirty solution I came accross some time ago is to use the power of mod_proxy - in this case you tell apache to forward all requests to a different port / URL whatever.
So I did the following:

• Set up CF 7
• Connect CF 7 to apache using the default way
• Set up CF 8 using the internal web server
• Enabled mod_proxy in apache (AddModule / LoadModule)
• Added the following lines in my virtual host configuration for a CF8 host:
  ProxyRequests on
  ProxyPass / http://127.0.0.1:8500/
• Worked!

Of course this is not a proper solution for deployment but for development purporses it's great. Using this way all apache modules and configuration possibilites (e.g. mod_rewrite, mod_deflate, access control) can be used and the internal webserver of CF is just the dummy backend.

No ugly JRUN error message on ColdFusion startup any more

Wouldn't it be nice to get rid of the ugly JRUN startup error message? Here (bloginblack) I have found a way to accomplish this - you just need to add a new line in the connector configuration in the apache configuration. I will give it a try in the next few days.

ColdFusion 8 / Apache 2.2.3

I recently set up a new testing server using the latest debian etch release. During the setup, apache 2.2.3 has been installed as default webserver and to make ColdFusion 8 (and maybe 7) running with this release, the following steps are necessary:

• Install CF8 using the built-in webserver
  
• Open /etc/init.d/apache2 with vi
• Note the paths set by APACHE2 / APACHE2CTL
• Now execute the following command:

• /opt/coldfusion8/runtime/bin/wsconfig -server coldfusion -ws Apache -bin /usr/sbin/apache2 -script /usr/sbin/apache2ctl -dir /etc/apache2/ -v
• bin = APACHE2 variable
• script = APACHE2CTL variable
• Open the httpd.conf and add AddHandler jrun-handler .cfm .cfc .cfswf.jsp .jws to the last line (.cfm / .cfc might be missing). Found this solution here.

Done!

Access other application scopes

In one of our projects, it was necessary to access application scope variables stored in a different application than the current one for security reasons. After some research I came accross this posting, which covers the technique to do exactly this: http://www.lynchconsulting.com.au/blog/index.cfm/2006/10/23/Hacking-the-application-scope-in-CFMX

var oApp = createObject("java","coldfusion.runtime.ApplicationScopeTracker");
var applications = oApp.getApplicationKeys();
Of course, this is an undocumented feature and who knows if it will still work in the next CF version, but it's good to know that it is possible ;-)

ORM - or why did I spend so much time with writing SQL by hand?

Some months ago I started playing around with ORM tools for ColdFusion - the two well known solutions in this area are Reactor and Transfer.
Until now most SQL code in our CFMX applications is written by hand or by a small component called "autoSQL", which is more or less a very stripped down version of an ORM tool. But as time goes by, it become more and more necessary to bring some standards in here and I am glad that I have found now a very good solution with Transfer.

If you need some arguments for using ORM, try these:

• Reduce time to write SQL
• Ability to switch to a new RDMS without touching the code
• More readable DB code
• Focus on the real important issues in your application (workflow, security)
  

How does the whole thing work?
First of all, you have to create some XML files which document your database and table structures. Transfer follows an package/object approach so that all the tables can be managed in an even more readable way.

The second step already happens in CFML - you play around with your data! Depending on the task you want to fullfill, several methods are available (Update, Save, Delete).

An example:

cfset var a_transfer = application.beanFactory.getBean( 'ContentTransfer' ).getTransfer()
cfset a_new_item = a_transfer.new( 'contacts.contact' )
cfset a_new_item.setentrykey( CreateUUID() )
cfset a_new_item.setname( arguments.name)
cfset a_new_item.setdescription( arguments.description )
cfset a_new_item.setdt_created( Now() )
cfset a_transfer.save(a_new_item)

The best way is to start with the included Blog demo application - there the whole concept is shown in a very good way. Take a look at the available presentations as well to gain a better understanding of the basic concepts.

Protect XML configuration files using .htaccess

In most of my projects, I create a mapping called /configurationxy to point to the configuration files (e.g. transfer XML files, mach-ii configuration and so on). This directory is not located under the webroot so no direct access is possible at all.
If this is not possible (e.g. due to shared hosting), never forget to protect your .XML files from being viewed and downloaded using e.g. the .htaccess feature of apache. In this case the file will be readable by the system itself but no user will be able to download the file. See an example configuration here.

Mach-II Framework 1.5 has been released

A new version (1.5) of the great open source MVC framework Mach-II has been released for production use. The most important change for me in this release is the introduction of XML includes, so that the main configuration - XML will stay clean.
To be honest, the learing curve of this framework was quite high for me in the beginning but now I don't want to miss the system any more.
One important note: Never put the business logic in the model components. I made this mistake in the beginning because it looked so damn easy but now I have created a sub directory named "cfc" and in this directory all the business logic is done.
So the perfect combination for me at the moment for a new project is:

• Mach-II framework
• ColdSpring for wiring CFC together
• TransferORM for DB access
• jQuery for interface operations

The perfect team for the next cool Web 2.0 application ;-)

escape vs encodeURIComponent

As long as you pass on only A-Z, 0-9 and so on to javaScript functions, encoding with escape is the proper method. As an Austrian company we have to deal with umlaute (special characters) a lot, however. Therefore in this situation encodeURIComponent is the best choice. Here you can find a good explanation and some examples concerning the differences.

Upload files to Webserver using WebDAV

WebDAV is a cool alternative to FTP and due to it's HTTP-basis, the typical port / passive problems of FTP can be ignored. Since version 7, ColdFusion support a huge range of the WebDAV commands, including TRACE, OPTIONS and so on.

To put a file to an enabled server, you simply have to use the following code:

cfhttp method="PUT" url="http://Server/Directory/Filename" username="username" password="password" throwonerror="true" cfhttpparam type="header" name="Content-Type" value="content/type"
cfhttpparam type="file" name="file" file="#a_str_full_path_of_filename#"
/cfhttp

Don't forget to check the cfhttp response for possible errors!