Subversion access without Apache: svn+ssh

I was always used to access subversion repositories through https, may it have been my own repositories or projects hosted on e.g. sourceforge.

By chance I realized that there is a much better, easier and more secure way to do that: svn+ssh

Just enter e.g. svn+ssh://username@subversionhost/repository as remote location and eclipse will automatically create a tunnel for you to this location and you don't have to run an apache on that host any more!

ANT is working fine with this type of location as well (although you might need to update some libraries) and I think ssh-compression is enabled because files will be transferred faster than with the default http(s) transport.

Root + Update HTC Desire to 2.2 the simple way

I've switched to a HTC Desire as my main phone some time ago and despite the fact that Froyo (Android 2.2) is around now for quite some time I have not received an update (Software update always told me that there is no new version available for my phone although I own an unbranded and unlocked model). The big deal with 2.2 is the WiFi hotspot mode for me, a feature I was using on a regular basis on my iPhone (Tethering) when I'm on the road.

There are several discussions (fire up Google) on this topic and the advice is to root the device and update the system with a custom ROM. While I'm a fan of playing around with devices and programs most of the time I'm not really happy with doing that with my phone. Why? I just want to use it for phone calls, check my emails, use some apps and browse the internet. No need to apply some specials hacks to my device.

Guess what - it's not that easy to find a firmware that's just doing that, finally I came accross this Pre-rooted Stock Froyo ROM, it's pretty much the default HTC firmware and updating to this version is really simple (You can download the original HTC firmware as well as it seems).

Just follow these three steps:

• Root your phone using unrevoked3
  No mess with gold cards (at least for me), just download the app and run it (Further Instructions)!
  
• Download the ROM from the source above and place it on the SD card
  Connect your phone using the USB cable and mount your SD card as drive
  
• Get ROM Manager from the Google Market (it's free)
  Click on "Update ROM" in the program and select the ZIP file which you've just copied to the SD card - the program will reboot your phone several times and perform the update.

The update procedure took me 20 minutes, the research 2 hours I guess ;-)

Most important, now I'm able to enjoy Froyo and the mobile hotspot feature.

A message towards Google - Please offer pre-build default Android versions which can be applied to any phone without any hassles using a simple "update" button like in iTunes, I'm sure most of the people are interested in a genuine platform and not a provider/vendor branded one with "special features".

Native hashchange event - Good bye polling

I just discovered that there is a native HashChange event in the latest version of several browsers (FF, Webkit, IE) - this way navigation solutions working with the hash at the end of a URL work without nasty timers.

To add the event, simply use:

if ( window.addEventListener ) {

window.addEventListener( 'hashchange', function () { checkHashChange(location.hash) }, false );

} else if ( window.attachEvent ) {

window.attachEvent( 'onhashchange' , function () { checkHashChange(location.hash) } );

}

Take a look at these samples as well. In real world applications you still will need to use a combination of both methods (polling + events).

Concerning the support of the feature in various browsers, please click here for a comparison chart.

Int( boolean) will return 0 or 1

Nothing special, but that's cute ... Int( boolean ) will return 0 or 1.

This comes in very handy especially for arguments with the type boolean where the corresponding field in the database is a tinyint.

CFIMAGE: Compile great looking artist images

I am really excited about the features CFIMAGE provides in railo/ColdFusion. In recent times, I've started doing advanced stuff (read: more than resizing images) and one of the thing I've really enjoyed was compiling acool artist background images from several images - below you can find three samples (Kayne West, Evanescence and Goldfrapp). Basically, it's just a simple combination of ImageRotate, ImagePaste and ImageRead ;-)

Sources: http://www.flickr.com/photos/96679304@N00/176295962, http://www.flickr.com/photos/11671827@N00/522201514, http://www.flickr.com/photos/87258901@N00/2951288353, http://www.flickr.com/photos/44124348109@N01/2834953166

Facebook Like Button will double your page hits

Nothing to be afraid of - while implementing the Facebook Like Button for tunesBag, I found out that every hit to a page with the "Like" - Button will be responsible for a second hit by the Facebook bot (facebookexternalhit/1.0)

UrlEncodedFormat: Encoding space characters - plus (+) vs %20

I came across an issue with UrlEncodedFormat() today - signatures calculated by an CF9-based system always appeared to be invalid and as it seems ColdFusion is ignoring the standards when it comes to encoding of form posts (application/x-www-form-urlencoded).

The document describing the standard on W3 clearly states:

application/x-www-form-urlencoded

This is the default content type. Forms submitted with this content type must be encoded as follows:

1. Control names and values are escaped. Space characters are replaced by `+', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by `%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').
2. The control names/values are listed in the order they appear in the document. The name is separated from the value by `=' and name/value pairs are separated from each other by `&'.

When you perform an UrlEncodedFormat( ' ' ) on CF, it will return %20, the same call on railo will return "+" as described in the article above.

I solved the issue by simply replacing %20 with +.

Fixing "An invalid XML character (Unicode: 0x0) was found in the element content of the document."

I had some issues recently with storing data in a database as WDDX package and later reading from database again ... as you know WDDX is XML based so CF/railo are using the built-in XML parser to read the data.

In this case special characters in the original content can lead to an error message like "An invalid XML character (Unicode: 0x0) was found in the element content of the document." - the solution is to escape the control chars in the string before calling CFWDDX, e.g. by calling

"REReplace(thisXML,'[\x0]','','ALL')"

I found this tip in the Adobe Forums.

Silent Setup of Sun JDK (Linux, Debian)

I was looking for a way to install the latest Java Runtime Environment without any user interaction and came across this posting

Here's a tips I found trying to do the same as you :
1. Create an answers.txt file with the word yes inside
2. Launch fhe following command (assuming you want to install jdk-xx.bin)
./jdk-xx.bin <>/dev/null

Works like a charm!

Ant & SCP on Mac

Just spend a few minutes involuntarily on fixing the SCP support for ANT on Mac ... a library is missing ("Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.Scp was not found.") and you have to copy a second file from the binary distribution, check out the howto I found on the web!

Important hint: Don't download the latest binary distribution, check which one you're running (ant -version) before doing that! Otherwise you'll likely receive error messages again.

NYCFUG - I was there ;-)

Some nice memories of my stay in NYC this summer - a picture with the famous CF blogger Ben Nadel at the NY CFUG! Two thumbs up!

Amazon EC2 goes Lorem ipsum

"Lorem ipsum" is the typical dummy text so I was quite surprised to read it on the Amazon EC2 Console (Setup wizard for a new instance) just a few seconds ago (see screenshot below). I guess I was just using the interface during an upgrade ... at least it reminds me a little bit of the pain during my Latin lessons in school ... ;-)

Add missing Array.indexOf to Internet Explorer

When dealing with JSON it comes in handy to get the position of a certain string in an array, image an array with column names returned by a SerializeJSON'ed CFQUERY.

Internet Explorer does not support this function, but just add the following snippet and you can use Array.indexOf( 'TITLE' ) in any browser without built-in support as well (We add a new function to the Array prototype!)

// make sure indexOf is supported
if(!Array.indexOf){
   Array.prototype.indexOf = function(obj, start){
       for(var i=(start||0); i<this.length; i++){
           if(this[i]==obj){
               return i;
           }
       }
   }
}

Apache, Railo & mySQL Troubles after Upgrading to Snow Leopard

Odd. The upgrade to snow leopard just killed by Apache Resin Connector and the vhost config - that should not happen (read other reports here). So make sure you backup your vhost configs before upgrading. For the connector, I had to re-download XCode in order to build the connector ... (700 MB in total, take care!).

The mySQL server wasn't starting as well, a symbolic link seems to be broken after the update, see here. Furthermore, some programs like Cyberduck (SSH) and Nambu (twitter) aren't working any more. To be honest, a quite disapointing update so far.

Setup Mac OS X + Apache + Railo

A fews days ago I upgraded to a Macbook Pro and I already spent almost two days on reinstalling software, copying documents etc etc (An upgrade is an opportunity for me to get rid of old software and ballast).

Today I started with the Apache / railo setup using a guide by Luid Majano (I had to download the Apple XCode Setup file (almost 1 GB) in order to compile the connector for Apache and reset the httpd to 32 bit instead of 64 bit).

Further problems I had to solve:

• Read how to fix the "Apache Error: Client denied by server configuration" error (very restrictive configuration makes sense on real life systems but on dev systems ...?)

• I wasn't able to access my files until I executed a chmod a+x on every directory in the path to the webroot (apache log [error] [client ::1] (13)Permission denied:)

Now it's working like a charm.

Using the H2 database in ColdFusion

Railo is delivered with an embedded database called H2. The description on the homepage says:

Welcome to H2, the Java SQL database. The main feature of H2 are:

* Very fast, open source, JDBC API

* Embedded and server modes; in-memory databases

* Browser based Console application

* Small footprint: around 1 MB jar file size

Such a database fit's perfectly for a current task at tunesBag - the streaming servers should store some data on their own machines and mysql or postgresql just would be overkill for this task. So I decided to give H2 a look and it looks pretty nice.

On my local machine I'm working with ColdFusion 8 as well - the configuration to add H2 support for CF is very easy, just follow the following steps:

• Download the latest version of H2
• Drop the h2*.jar file in the WEB-INF/lib directory of your ColdFusion server (directory depends on your type of setup)
• Restart CF
• Add a new datasource
• JDBC URL = jdbc:h2:file:%Path to Database file on your disk%
• Driver Class = org.h2.Driver
• Driver name = default
• User name = sa

That's it - the files holding the data will be created automatically for you. Take a look at the H2 tutorial in order to find out more (e.g. server mode etc).

Railo - Compatibility issues with scopes (URL etc)

I'm migrating my development environment to railo and found some issues that affected my code.

One of the most important ones is that railo always references the scope name when you use URL, FORM etc no matter if a local variable with this name exists or not. To give you an example, run this code in CF and in railo:

<cfparam name="url.step" type="string" default="1" />

<cfset a_call = PrintURL( 'http://tunesBag.com/' ) />

<cfdump var="#a_call#">

<cfscript>
function PrintURL( url ) {
return url;
}
</cfscript>

To make railo behave the same way, you've to add a full reference to the function -

return arguments.url

In ColdFusion you'll see http://tunesBag.com as output, in railo the URL structure. Some further explanations concerning the scopes and this "bug" can be found here (Adobe Livedocs) and here (Railo Bugtracker)

Having fun with railo: CFQUERY supporting complex objects

I just came across a cool feature of railo - it's possible to set query columns to complex objects, e.g. a java object or a structure.

An example:

cfset q = QueryNew('data');
querySetCell( q, 'data', { firstname = 'John', surname = 'Doe' }, 1);

cfdump var="#q#"

You can access the structure by using q['data'][1]!

cool!

Important: This complex data won't survive a query of queries operation, in this case the simple classname will be shown in the column.

CFFEED not usable due to missing timeout property

I've always used CFFEED in order to read newsfeeds (e.g. display recent blog postings on the homepage); now we had serious problems with non-responding feeds - and as it seems there is no workaround (see posting by Ben Nadel as well). As solution I decided to go back to cfhttp and simple parsing of the XML document.

"Corrupt table" errors (ColdFusion 8, mysql)

We have to do a lot of caching in order to keep the performance of tunesBag.com, so cached queries are very important for our website. During the last two days a lot of "corrupt tables" errors showed up in the log and first I thought about a mysql database issue of course. It's a bug in ColdFusion, however, so in case you run into the same error use this hotfix: http://kb.adobe.com/selfservice/viewContent.do?externalId=kb402583

I've updated our live servers and I hope we won't see this error again.

Enhance performance of Joins / Queries: Make sure all columns share the same collation

At tunesBag.com, we're storing most of the data in UTF-8 / Unicode because we've customers from all over the world. Some internal data like userkeys, entrykeys etc are stored in latin1 because it needs less space. I recently came accross a query which was very slow without any obvious reason and after some research I found out that in a left join, the column A from table A has a different collation than column B from table A.
I changed the collation for column B to the same as column A - and the query executive time dropped from about 1300 msec to 100 msec!

So always make sure you're using the same collation if possible when performing joins!

JavaScript variables set in onclick - different behaviour in IE / FF

I recently discovered that the behaviour of internet explorer and Firefox is quite different when it comes to settings variables in an onClick event - in Firefox these variables will become global variables, in IE they are just "local" and will nur be available for other functions.

An example:

a onClick="variableA = 'peter';CallFunction();return false;" ...

function CallFunction() {
alert(variableA);
}

In FF, an alert box will show "Peter", in IE you'll receive an exception.

How to solve that? Simply declare variableA on top of your JavaScript file, this way IE will modify the global variable.

var variableA = '';

Railo - important path placeholders

I'm playing around with railo, a full featured alternative CFML engine and so far, I'm very impressed. During testing my existing applications with this new environment I had to add some custom mappings in the administration (by the way, there are two admin engines, the first one is server-wide, the second one just for the current webroot).

When creating these custom mappings, there are some very interesting path placeholders you can use:
{railo-web}: path to the railo web directory typical "{web-root}/WEB-INF/railo"
{railo-server}: path to the railo server directory typical where the railo.jar is located
{temp-directory}: path to the temp directory of the current user of the system
{home-directory}: path to the home directory of the current user of the system
{web-root-directory}: path to the web root
{system-directory}: path to thesystem directory

This way you can create mappings without using the full path to the directory - very cool!

Create & Connect to databases (mysql, SQLite etc) on the fly with ColdFusion

Working with databases in ColdFusion is very easy - you have to set up your datasources in the CFAdmin (or by code using the CFAdmin API) and you can start writing SQL using CFQUERY. But what if you want to connect to a database on the fly without creating a datasource?
In this case it comes in handy that ColdFusion is a Java-based product, so you can use all the power JDBC is offering. At the online music hub tunesBag I recently had to create SQLite databases on the fly - it's quite easy in fact as you can see below:

1) Add the JDBC Driver (or use an existing one)
You can use any JDBC driver, e.g. mysql, MS-SQL etc
In my case: Download the SQLite JDBC Driver from zentus.com and place it in wwwroot/WEB-INF/lib (sqlitejdbc-v053.jar)

a_sqlite = createObject( 'java', 'org.sqlite.JDBC' )
a_prop = createObject( 'java', 'java.util.Properties' )
db_filename = '/tmp/sqlite_' & CreateUUID() & '.db'

2) Write the code (this is an example for sqlite)

a_conn = a_sqlite.connect( 'jdbc:sqlite:' & a_db_filename, a_prop.init() )
a_statement = a_conn.createStatement()
a_res = a_statement.execute( 'BEGIN;')

Create the table
a_res = a_statement.execute( 'CREATE TABLE test (id INTEGER PRIMARY KEY, firstname TEXT, surname TEXT);');

Insert data
prep = a_conn.prepareStatement( 'INSERT INTO test (firstname,surname) VALUES (?, ?);' );

prep.setString( 1, 'John' );
prep.setString( 2, 'Doe' );

// you can add multiple inserts ...
prep.addBatch();

prep.setString( 1, 'Max' );
prep.setString( 2, 'Mustermann' );
prep.addBatch();

// insert!
prep.executeBatch();

// save the changes
a_res = a_statement.executeUpdate( 'END;');

// quit
a_conn.close();
a_conn = 0;

Where to go from here
For more details, check out the official JDBC documentation provided by SUN!

mySQL - cfqueryparam - Comments - "No value specified for parameter"

I always try to add comments to complex queries against our mysql server and in order to see the comments in the debug output as well, I'm always using the /* comment syntax */ offered by SQL. The other day I added some comments to a rather complex query again and suddenly I received a strange error message all the time telling me "No value specified for parameter 4 / SQL State 07001".
After some playing around I found out that you should now use question marks in the SQL comments - it seems as CF or mysql is interpreting this char and assuming a parameter, even if it's contained in a comment.

New ColdFusion primer / tutorial

I've just finished a brand new tutorial for Adobe ColdFusion / CFML - in this primer I cover the basic server environment, database access, SQL, flow control, error handling and much more. Check it out here.

Not all CGI variables show up using CFDUMP

CGI variables must be something special - it is not possible to show all cgi variables when doing a cfdump. I just came across this bug when trying to use a custom #404 page where apache provides the original page URL in the CGI variable REDIRECT_URL.
I am just using CGI.REDIRECT_URL now and it works.
The interesting thing is that StructKeyExists(CGI, blabla) does not work at all - the check will always return true.

VARing cffile and the result variable

I am using cffile to upload incoming files and of course in my component I "var" the cffile variable at the top of the function (like one should do with cfhttp etc):

The problem is that somehow the cffile scope will stay zero - until I added the property "result" to the cffile tag saying the result should be stored as "cffile" - so the whole call is:



Now it works again!

Web Developer's Handbook

This endless list is like a Web Developers's Handbook at it's best ... a lot a page with stuff you already know but some interesting news as well

Apache: Disable HostNameLookups

After an upgrade to our infrastructure (see a picture here), the delivery of content through apache was incredible slowly - the reason was that we moved our DNS servers to an external location and had no internal server for this task any more.
After some research we found out that disabling the host name lookup (HostNameLookups = Off) helped to speed up the pages delivery. Now our analyzer will perform the lookup task and we're happy again ;-)

JRUN crashes after changing the hostname of a machine

Yeasterday we updated our system and renamed a server from www-4 to www04 in our internal DNS system. After that, this server started to deliver pages very slowly (with a very low load, however) and after some reasearch we found out that chaning the hostname was responsible.
So make sure your internal hostname and the hostname set in /etc/hostname is the very same!

MD5 Hash values: Be aware of uppercase/lowercase

These days I had to deal with a web service which needs a parameter as MD5 hash. I played around with for some time but authentification always failed, I did everything according to the documentation, however. Then I found out that the other service is checking the hash value case-sensitive, so I had to add a lcase to the string.
Some further research shows that almost every other service I know produces lowercase only hash values, ColdFusion produces uppercase values. Be aware of that! ;-)

JVM heap errors: Check every cffile / cfhttp request

This weekend I ran into trouble because the JVM (too small heap size) was crashing several times and after some research and debugging I found out that reading a 80 megabyte file with cffile was the reason for all the pain.
So be aware of this and do the following:

• Check the size of uploaded files (by performing checks with the size given in cffile or using a routine like that).
• Before you request any unknown website using cfhttp, do a HEADER operation and check out for the size (cfhttp.responseHeader, Content-Length) or add Content-Range headers to the request in order to allow only a certain content size
  

ColdFusion / MacOS 10.5 / WebConnector: Not again!!

Every update of apache is a real pain for me and maybe most of the ColdFusion community. Each time the server engine is updated, the WebConnector has to be updated as well. In the current case, I decided to update my Mac OS finally to Leopard (10.5) and - bingo - with the apache update from 1.3 to 2.2 the whole connector stuff isn't working any more.
One solution might have been again to download the whole connector sources (XCode) and compile the connector myself - some hours work plus about 1-2 GB downloads for an about 90kb file.
The other solution: Follow these instructions and download the file offered for download.

Attention: INNER JOIN is now the default in transfer

With the update to the latest release of the CF - based ORM system Transfer 0.6.3. , the default JOIN behaviour has been changed to INNER JOIN - I was wondering why some of my queries started to return a much smaller number of records at once ;-)
The solution was to write the full LEFT OUTER JOIN statement instead.

Very uncool - no implicit structure creation within function calls

I am using more and more features provided by CF8 as we migrate our servers to this new system. One feature I love is the implicit creation of structures without the need of calling StructNew() all the time.
So - a major downside is that you cannot create structures using this way within functions calls ... very bad. An example:

a_component.StoreInformation( key = arguments.key, data = { firstname= a_str_firstname } )

Very uncool, same thing noticed by Ben

Reset ColdFusion Administrator password

Today I wanted to edit a datasource name in the ColdFusion Administrator but I did not remember the password on this development server (hell, where are the nice little yellow post-its ;-)).
Here you can find a nice solution how to achive a password - reset.

XML vs JSON: Incredible performance differences

I admit - I am a real fan of XML and try to use this format whenever it is possible. The reason is simple: It's a clear format and very easy to exchange data with other applications (also with applications not running in a browser).
During the finalization of a project we came across several performance problems with a huge dataset and I tried to move the data interaction to JSON.
The result: The time of parsing a XML with jQuery (loop over the element using each) was up to 100 times higher than doing a simple JSON request (According to the results of the firebug console.timer output). No special treatment, just looping over the records and building an array in javaScript using the default jQuery way.

So, I hope in future times XML parsing will speed up with faster computers (although I think there will be not much gain in the near future because browser XML parsers just use MSXML or Xerces right now).
If you do not have ColdFusion 8 (which offers built-in JSON support), you might take a look at this library.

20 ways to secure your apache server

I can only recommend to go through the points of this posting if you want to secure your apache webserver.

Easy way to run several ColdFusion instances using one apache server

As you might know, in the server configuration JRun is the J2EE server in the background of CF and so in the apache configuration you will notice that the whole connector stuff is done using JRunConfig* directives.
As long as only one version of CF is running using apache as webserver everything is fine, it starts getting complicated when you decide to run e.g. 7 and 8 on one machine using one apache instance. Here you can find a solution how to set it up with different JRun ports which I did once as well (took me hours to find out all these stuff ;-)).
A very nice, quick and dirty solution I came accross some time ago is to use the power of mod_proxy - in this case you tell apache to forward all requests to a different port / URL whatever.
So I did the following:

• Set up CF 7
• Connect CF 7 to apache using the default way
• Set up CF 8 using the internal web server
• Enabled mod_proxy in apache (AddModule / LoadModule)
• Added the following lines in my virtual host configuration for a CF8 host:
  ProxyRequests on
  ProxyPass / http://127.0.0.1:8500/
• Worked!

Of course this is not a proper solution for deployment but for development purporses it's great. Using this way all apache modules and configuration possibilites (e.g. mod_rewrite, mod_deflate, access control) can be used and the internal webserver of CF is just the dummy backend.