GoDaddy as a trusted certificate authority

GoDaddy has not as of yet ascended to the ranks of a default trusted certificate authority in the Java Security code. You have to make this happen manually.

I was getting an exception when attempting to consume a web service over SSL. Here is the exception:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target Message: ; nested exception is: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The fix for this problem was to add GoDaddy as a trusted CA. Here are the steps I took:

1. Download gd_intermediate.cer from here.
2. I put this file in %JAVA_HOME%\jre\lib\security. You can put it anywhere you want.
3. Run the following command at a command prompt:

C:\Program Files\Java\jre1.6.0_02\bin>keytool -import -trustcacerts -alias godaddy-cert -keystore ..\lib\security\cacerts -file ..\lib\security\gd_intermediate.cer

4. You will be prompted for a password. If you have not changed the password, it will be "changeit".
5. You will then get the following message if all is successful - "Certificate was added to keystore".
6. Restart Tomcat.

Monitor MySQL and restart it if it has stopped

I had a need to monitor a MySQL database instance running as a Windows service and then restart it if for whatever reason it had stopped or died unexpectedly. From within my java web application, I created a Quartz job and scheduled it to run once per minute. I manually set the connection timeout to 10 seconds for the jdbc driver so as not to have to wait 30+ seconds if the MySQL instance were down. Here is the code that runs as part of the Quartz job:

Connection con = null;
try {
  DriverManager.setLoginTimeout(10);
  class.forName("com.mysql.jdbc.Driver");
  con = DriverManager.getConnection("jdbc:mysql://[server]:[port]/[dbname]", "[db user]", "[password]");
  // If we did not get a connection, the db must be down. Start it up again.
  if (con == null) {
    Runtime rt = Runtime.getRuntime();
    try {
      rt.exec("net start mysql");
    } catch (IOException e) {
      e.printStackTrace();
    }
  }
} catch (Exception e) {
  // If we did not get a connection and an exception is thrown, the db must
  // be down. Start it up again.
  if (con == null) {
    Runtime rt = Runtime.getRuntime();
    try {
      rt.exec("net start mysql");
    } catch (IOException e2) {
      e2.printStackTrace();
    }
  }
  e.printStackTrace();
} finally {
  // Clean up.
  try {
    if (con != null) {
      con.close();
    }
  } catch (SQLException ignored) {
  
}

This will only work if the instance of MySQL is running on the same box as the java web application. Also, the "mysql" in the command "net start mysql" is the name of the Windows service that I am wanting to start. One last thing, in my case the java web application was running under Tomcat. I had to make sure that the Tomcat Windows service was started with the same Windows account as the MySQL Windows service.

If you have better ways of doing something like this, please share.

HelpMeFinancial.com

We are designing a new financial portal HelpMeFinancial.com. It will be a place where we can use all of our calculator and article content, as well as other financial related information. The plan is to be "live" sometime in Q1 of this year. We plan on making it an ad-driven portal as well.

There are numerous financial portals on the internet. Our "edge" will be that this site will be centered around our expertise - FINANCIAL CALCULATORS. Whenever anyone needs a calculator, they can come to helpmefinancial.com and easily find the one they need. We will make it easy to understand and interpret the results. We will provide links to similar calculators or other calculators that may be able to provide additional information.

The site will be available in English and Spanish.

Testing of financial calculators in a blog

I wanted to try out some new "blogger-friendly" financial calculators available at CalcXML.com. For example, here is the Loan Payment calculator.

Financial Calculators/CalcXML

In January of 2007, I became 50% owner of Financial Calculators, Inc., after buying out one of the two partners of the company. The other partner, Chris Brockbank, and I also created a new company called CalcXML, LLC. Both of these companies provide over 100 online financial calculators in 11 categories to the financial services industry, including large portals (Yahoo and AOL), insurance companies (Mutual of Omaha, Guardian Life, Jackson National, National Life), banks (Wells Fargo, Everbank, First Horizon) and other financial service companies (LendAmerica.com, Experian, etc.)

Financial Calculators, Inc. provides our legacy calculators where we host the input and output pages along with the calculator engines themselves. Clients have the ability to customize the look-and-feel, add their own logo, etc. and then put hyperlinks on their web site to access the calculators.

CalcXML, LLC provides our new XML web service based calculators. Clients have full control over the look-and-feel of the input and output pages and can host them on their own servers, making behind-the-scenes web service calls to our calc engines for the calculator results. As the first and only calculator company to offer such a solution, we have had many sales opportunities this year.

Page hijacking

We recently suffered a page hijacking incident on our servers. The symptoms were that when someone browsed to one of our pages, they would be redirected to some other page. It took us some time to figure out how it was happening. We finally stumbled upon a post on a forum that helped us solve the problem. I thought I'd include a link to that post here to help others find it if they happen to find my blog post. This post describes exactly what was happening to us and provides a solution to the problem.

http://www.webmasterforums.com/windows-web-security/5612-page-being-hijacked.html

The solution is on the 3rd page. If for some reason that link is gone or does not work, I copied it's contents to here:

http://www.whitesandsolutions.com/PageHijacked.rtf

Here are some keywords that someone may use to find this post:

c:\boot.vbs, mm.exe, nb88, d99_temp, ad.js, coon.js

TemplateMonster Template Installation

After having used XAMPP to install Apache, MySQL and PHP, I am now ready to install a web site template from TemplateMonster.

I purchased and downloaded a template and copied the unzipped files to ../htdocs/shop. During the install of the template, which is just like the install of the dummy catalog that comes with osCommerce, make sure to check the box to save sessions to the database, not files. I had problems with files. It complained about a tmp directory not existing. I created one but it still gave me problems.

I also went in and changed the directory security in my httpd.conf file to protect the admin directory. See previous post for details.

There were some sql errors after installing the template. I had to modify some of the queries to make them work. I found help on the osCommerce site.

First change is here. Do what is suggested in post dated 11 Nov 2005 17:16:06.
Second change seems to be related to MySQL4 vs MySQL5. I have MySQL5 running. In order to fix the queries, I had to find out which .php file(s) the queries were in. I couldn't search the .php files using Windows Explorer because Explorer is stupid. I had to download a little utility called XP_FileFilter.exe from here, that helps you enable searching for files containing text with files that have "unregistered extensions".

I did what was suggested in the post dated 15 Jun 2006 07:18:13 of the second change link above. The change needed to be made in the \htdocs\shop\index.jsp file. (Thank you XP_FileFilter!)

Now the admin pages...
I browsed to the admin site, entered my username and password, and got this:

Fatal error: Cannot re-assign $this in C:\Program Files\xampp\..\includes\classes\upload.php on line 31

Fix is here. Change $this = null; to unset($this);

Other sql query fixes
modified products_new.php - changed from clause to have inner join instead of comma
modified index.php - changed all from clauses to have inner joins instead of commas
modified index.php - changed a reference from p2c.products_id to p.products_id in one of the queries. p2c was not an alias in the query.

osCommerce 2.2 and PHP 5.2

osCommerce 2.2ms2
PHP 5.2.0RC2-dev

Driven by the desire to be able to use slick web site templates provided by TemplateMonster, I discovered I would need to install osCommerce. TemplateMonster has web site templates that run on osCommerce 2.2.

osCommerce is an online shop e-commerce solution that offers a wide range of out-of-the-box features that allows online stores to be setup fairly quickly with ease and is available for free as an Open Source based solution.

osCommerce can be installed on any server where a web server is running that has been installed with PHP.

NOTE: The following few paragraphs describe my attempt to get this working by manually installing each app. I scrapped my manual installation of these apps and went with XAMPP's installation instead. The details are further down in the post.


So to get started I installed a PHP Hypertext Preprocessor on my server running Apache 2.2. After two failed attempts with PHP 4.4.2 and PHP 5.1.4, I discovered that I had to use PHP 5.2.0RC2-dev, which is the only version that runs on Apache 2.2. I'm hoping for a release version soon.

I unzipped the zip file to c:\php and followed the directions outlined in the install.txt file.

Highlights:
Added "c:\php" to my PATH environment variable.
Added to Apache httpd.conf:
LoadModule php5_module "c:/php/php5apache2_2.dll"
AddType application/x-httpd-php .php
PHPIniDir "C:/php"

Tested the PHP installation with a simple test.php file that I stuck in my default Apache doc folder:
C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\test.php

The contents of test.php are:
<?phpinfo()?>

Then just browse to http://localhost/test.php and you should get a page showing a bunch of system info. NOTE: I had to use the php.ini-dist file instead of the php.ini-recommended file to get test.php to display anything. I didn't compare the two to find out why.

Installing osCommerce
Looks like there are problems getting osCommerce to run with PHP5.

The milestone release that I have claims to have fixed some PHP5 issues. Continuing with installation. Follow directions in documentation.pdf.

php.ini - turned register_globals On

Step 2 of osCommerce install does not work.

Added "php_value register_long_arrays 1" to htdocs/catalog/.htaccess file.
(didn't help)

XAMPP to the rescue!!

After unsuccessful efforts to get it working, I decided to scrap it all and use XAMPP, an easy to install Apache distribution containing MySQL, PHP, Perl and a bunch of other apps, including an add on install for Tomcat.

I used the Windows installer version of XAMPP. Then I used the Windows installer version of the Tomcat add-on.

Other things:
Enabled innodb engine in mySQL because I wanted full transaction support in my databases. Modified ..\xampp\mysql\bin\my (no extension). Commented out:
#skip-innodb
Uncommented all the innodb lines after the skip-innodb line. I also increased most of the memory settings throughout the file since I have more the 64M on my machine.

Set jvm memory settings for Tomcat.
Modified ..\xampp\tomcat_start.bat. Added line:
set JAVA_OPTS=-xms256m -xmx256m

After installing Tomcat as a service as mentioned below, I ran ..\tomcat\bin\tomcat5w.exe to set memory options, since the service does not use tomcat_start.bat.

Enabled name-based virtual hosts.
Modified ..\xampp\apache\conf\httpd.conf. Uncommented lines:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_http_proxy.so
Include conf/extra/httpd-vhosts.conf

Modified ..\xampp\apache\conf\extra\httpd-vhosts.conf. Uncommented line:
NameVirtualHost *:80

Added in all my virtual hosts.

Used apache -S and apache -t at a command prompt to check syntax.

NOTE: I did have a problem because I copied the sample virtual host entries in httpd-vhosts.conf and then modified them. I assumed that @rel_logfiledir@ was some sort of macro and just left it in my virtual host definitions for the error logs. This caused apache to fail on startup with a message in the Windows Event Viewer as follows:

The Apache2 service terminated with service-specific error 1 (0x1).

After a bit of trial and error, I figured out it was the @rel_logfiledir@ that was causing the problem.

I went through the security section of the docs and secured everything as was suggested. I downloaded and installed mySQL Administrator and mySQL Query Browser from mysql.com since I am more familiar with those mySQL tools than the ones that come bundled with XAMPP.

osCommerce Installation
Unzipped and followed instructions in documentation.pdf.
xampp\apache\bin\php.ini - turned register_globals On
Everything installed fine following the instructions in the pdf and on screen.

Address Some Security Warnings
When I ran the default osCommerce catalog site, I had two warnings at the top. The first advised me to delete the install folder for security reasons. The second said:

Warning: I am able to write to the configuration file: C:/Program Files/xampp/htdocs/catalog/includes/configure.php. This is a potential security risk - please set the right user permissions on this file.

I just had to make the file read only.

I discovered that the osCommerce administration tools were wide open to anyone savvy enough to browse to the \catalog\admin URL.

Securing the osCommerce Admin Tools
Create an adminosc user in apache. The following command does this and asks for a password:

C:\Program Files\xampp\apache\bin>htpasswd -c "c:\program files\xampp\apache\passwd\passwords" adminosc

Add the following to httpd.conf to secure the admin directory:

<directory>
AuthType Basic
AuthName "osCommerce Admin Files"
AuthUserFile "C:/Program Files/xampp/apache/passwd/passwords"
Require user adminosc
</directory>

Email
Although XAMPP comes with Mercury Mail Transport, I didn't want to deal with configuring a mail server on my server, so I decided to just use my ISP's mail server for all email needs.

I was having trouble getting email to work using my ISP's mail server. I had to:

• Change php.ini's email settings (SMTP = mail.xmission.com). Left other settings commented out, except smtp_port of course.


• Changed the tep_mail function in \xampp\htdocs\shop\includes\functions\general.php as suggested here.


• Changed settings in osCommerce admin's E-mail Options page. Changed "E-Mail Transport Method" to smtp and "E-Mail Linefeeds" to CRLF since I'm running on a Windows box.

Tomcat
Configure Tomcat to run as a service. By default, the XAMPP installation provides batch files to start and stop Tomcat. I want to run it as a service so that when the system reboots, I don't have to login to start Tomcat.

To run Tomcat as a service, simple open a cmd prompt here \xampp\tomcat\bin and type:

service.bat install

Then open the Services control panel, edit the properties of the newly installed service and change startup type to "automatic" instead of "manual".

Changed the Logon user of the service to use a tomcat user account that I setup. Didn't want the Tomcat service running under an admin account for security reasons. I edited the tomcat folder security settings, giving full control to this user as well.

tomcat user

• Member of "Users"
• User cannot change password
• Password never expires

Virtual hosts with Apache and Tomcat

I wanted the ability to host multiple domains from one instance of Tomcat and also wanted to keep the domain name in tact in the browser address bar. I had already been hosting multiple web sites, but when the user would type something like "www.somedomain.com", I would have to redirect them to "www.whitesandsolutions.com/somedomain/do/home" or something like that. What I wanted was the www.somedomain.com to remain in tact.

After a bit of reading, I decided to install the Apache web server and use the virtual host functionality. Click here for the Apache virtual hosts documentation. Name-based virtual hosts were exactly what I needed. Making it all work correctly took a bit of trial and error. Here are some notes of mine. I may not have it configured in the best possible way, but it does work...

Install Apache Web Server 2.2
Install Tomcat 5.5.17

Here is my Apache httpd.conf file. All commented lines have been removed for brevity:

ThreadsPerChild 250
MaxRequestsPerChild 0
ServerRoot "C:/Program Files/Apache Software Foundation/Apache2.2"
Listen 80
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so
LoadModule asis_module modules/mod_asis.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule imagemap_module modules/mod_imagemap.so
LoadModule include_module modules/mod_include.so
LoadModule isapi_module modules/mod_isapi.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule userdir_module modules/mod_userdir.so
ServerAdmin your.name@yourcompany.com
ServerName 123.123.123.123
DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs"
<Directory />
Options FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Satisfy all
</Directory>
<Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs">
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<IfModule dir_module>
DirectoryIndex index.html
</IfModule>
<FilesMatch "^\.ht">
Order allow,deny
Deny from all
</FilesMatch>
ErrorLog logs/error.log
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog logs/access.log common
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin/"
</IfModule>
<Directory "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
DefaultType text/plain
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule>
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
NameVirtualHost *:80
<VirtualHost *:80>
ServerName 123.123.123.123
ServerAdmin your.name@yourcompany.com
DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs"
ErrorLog logs/error.log
</VirtualHost>
<VirtualHost *:80>
ServerName www.domain1.com
ServerAdmin your.name@yourcompany.com
DocumentRoot "C:/Tomcat 5.5/webapps/domain1/ROOT/"
ServerAlias domain.com
ErrorLog logs/error.log
ProxyPass / http://localhost:8081/
ProxyPassReverse / http://localhost:8081/
</VirtualHost>
<VirtualHost *:80>
ServerName www.domain2.com
ServerAdmin your.name@yourcompany.com
DocumentRoot "C:/Tomcat 5.5/webapps/domain2/ROOT/"
ServerAlias domain2.com
ErrorLog logs/error.log
ProxyPass / http://localhost:8082/
ProxyPassReverse / http://localhost:8082/
</VirtualHost>
<VirtualHost *:80>
ServerName www.domain3.com
ServerAdmin your.name@yourcompany.com
DocumentRoot "C:/Tomcat 5.5/webapps/domain3/ROOT/"
ServerAlias domain3.com
ErrorLog logs/error.log
ProxyPass / http://localhost:8083/
ProxyPassReverse / http://localhost:8083/
</VirtualHost>

If I remember correctly, the relevant changes I made from the version of this file as it was installed by default was to uncomment a couple of LoadModule lines and then add the virtual host lines at the bottom. I wanted to include the whole thing though for reference.

Once this was done, I then had to configure Tomcat. Here is my Tomcat server.xml file:

<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.storeconfig.StoreConfigLifecycleListener"/>
<GlobalNamingResources>
<Environment name="simpleValue" type="java.lang.Integer" value="30"/>
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />

</GlobalNamingResources>
<Service name="Catalina">
<Connector port="8080" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true" />
<Connector port="8009"
enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
<Engine name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>
</Service>
<Service name="domain1">
<Connector port="8081"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" acceptCount="100" connectionTimeout="20000"
proxyName="www.domain1.com"
proxyPort="80" disableUploadTimeout="true" />
<Engine name="domain1" defaultHost="domain1-host">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>

<Host name="domain1-host" appBase="webapps/domain1"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>
</Service>
<Service name="domain2">
<Connector port="8082"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" acceptCount="100" connectionTimeout="20000"
proxyName="www.domain2.com"
proxyPort="80" disableUploadTimeout="true" />
<Engine name="domain2" defaultHost="domain2-host">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="domain2-host" appBase="webapps/domain2"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>
</Service>
<Service name="domain3">
<Connector port="8083"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" acceptCount="100" connectionTimeout="20000"
proxyName="www.domain3.com"
proxyPort="80" disableUploadTimeout="true" />
<Engine name="domain3" defaultHost="domain3-host">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="domain3-host" appBase="webapps/domain3"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>
</Service>
</Server>

The Tomcat directory structure looks like this:



The rest is all standard stuff. Create your ROOT.xml context files, and deploy your web sites. I created an index.jsp file for each of my apps so that when the user typed in www.domain1.com it would redirect to www.domain1.com/do/home. My index.jsp file looks like this:

<%@ taglib uri="/WEB-INF/lib/struts-logic.tld" prefix="logic" %>
<logic:redirect href="http://www.domain1.com/do/home">;

And I added index.jsp to my welcome-file-list in my ROOT.xml context file:

<welcome-file-list>
<welcome-file>index.jsp</welcome-file>
</welcome-file-list>

Comments are welcome. If you have better, more efficient ways of setting this up, please let me know.

New Company - RemindMeCall

I started a new company with a friend of mine, Chris Brockbank, called RemindMeCall, http://www.remindmecall.com. It's been a blast getting it going. I developed the web site and the web application, and Chris has been testing the app and doing some marketing.

We have a number of people excited about the product. One company, Charmed Moments, has six of their top distributors using our system, in preparation for a big sales conference they are having in August. They're using it to place reminder calls to the guests invited to their parties. They anticipate it will boost attendance at the parties, and therefore, boost the sales at the parties. Kim Roylance, President and Co-founder, is hoping to create some compelling case studies from these six distributors over the next couple of months and then present it at the sales conference in August. This could mean hundreds of new subscribers for us!

There are two other large direct sales companies we will be speaking with in the next 3-4 weeks.

Another market for the system is the medical industry, including dentists, optometrists, chiropractors, etc. All of these professionals deal with appointments, and having even one "no-show" results in a loss of $100+ dollars. We have an optometrist in Idaho who is using the system and is planning on having his wife attend his next convention, setting up a booth, and selling the product.

She can do this because RemindMeCall has a great upside if you sell it: 25% commission, plus incentives to increase that up to 40%. And this is recurring revenue, meaning each month that your clients continue to use the system, you as a sales person receive your cut of their monthly subscription. It's easy to see how your monthly intake could just keep growing and growing!

There are so many possibilities for the use of this system. See personal uses and business uses.

I am currently working on providing a web service interface to the product, which opens the door for software vendors to integrate their systems with RemindMeCall's suite of services. All the software products used by doctors, dentists, chiropractors, optometrists, etc., could provide appointment reminders easily and effectively by integrating with our web service interface.

It'll be interesting to see how it all turns out.

Scrum

I finished reading the book Agile Software Development with Scrum by Ken Schwaber and Mike Beedle and discovered that there is a way to develop software that actually makes sense! Great book.

Having developed software for 13 years now for a few different companies, I've come across a number of different methodologies, each touting the reasons why they will work. One fundamental difference between all those and Scrum is that they all have adopted a "defined" process control model, whereas Scrum uses an "empirical" process control model to manage the development effort.

A defined process is one in which, given a set of inputs, you can pretty much guarantee a certain output. You can do this over and over again. This works great in the manufacturing industry. Years ago, someone or 'someones' thought this would work in software, and we've never really gotten away from it. The defined process can be used successfully when every piece of work is completely understood. A defined process can be started and run to completion with the same results every time.

How often is this the case with software development? ... never, yeah never. Software development is complex, unpredictable and never completely defined up front. This type of problem requires an empirical control model. An empirical process "provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs." (pg 25)

Another thing that Scrum fosters and promotes is the idea of self-organizing teams. Set a couple of goals for a team of 4-8 people to accomplish in 30 days and let them go for it. Check in with them each day to monitor progress and identify impediments (the empirical process). The team is 100% responsible for getting the job done and will organize themselves to be successful.

The Scrum process is essentially this:
1. Create a list of bugs, enhancements, features, etc. (The Product Backlog)
2. Prioritize the list.
3. Create a team and appoint a Scrum Master for the team who is responsible for enforcing Scrum practices.
4. Have the team select 30 days worth of the highest priority items in the list.
5. Begin the 30-day Sprint where the team works uninterrupted for 30 days.
6. Meet each morning for a 15 minute Daily Scrum Meeting to identify impediments, report on progress from the previous day, and identify what each team member will do today.
7. After 30 days, hold a Sprint Review where everyone gets to show off the executable product code they've created.

There's a whole lot more to it than this, so read the book! After you've read it and agree that it is a great way to development software, you can use ScrumTool (shameless plug), a great, web-based tool to help you manage your Scrum projects.

iDebtManager Goes Live

iDebtManager, which is Phase II of Ideal Financial Solutions' CashFlow Management System, has been completed and released for the general public.

iDebtManager allows users of the CashFlow Management system to stay up-to-date on the progress they are making in becoming debt free by tracking the balances on all debts being managed by the CashFlow Management System. All of the user's debts that are being managed by the CashFlow Management System can be viewed individually and payment amounts can be adjusted if desired. New debts can also be added to the system.

A Payoff Timeline is available which allows users to see when each debt is scheduled to be paid off as well as the progress they are making to become debt free. The Payoff Timeline gives the user the ability to compare how he is actually doing in his efforts to become debt free against the CashFlow Management System's proposed plan.

To see a demonstration of iDebtManager's functionality, visit the Ideal Financial Solutions' website and select the iDebtManager Demo link.

Running regular, periodic jobs with Quartz

Many applications have the need to run regular, periodic jobs/processes. Such is the case with the application I'm working on. One option is to write your own job processing engine, but why reinvent the wheel. Another option is to use classes provided in the Java SDK, java.util.Timer and TimerTask, but there are limitations here. For example, there is a very limited set of scheduling mechanisms and the timers don't use a thread pool.

I came across the Quartz Job Scheduler, an open-source project by OpenSymphony. What a great product! I have integrated it into two different Struts-based web applications and it was quite simple to do.

There are a couple of ways to integrate it. One way is by adding a reference to the Quartz servlet in your web.xml file. I did have a couple of problems with this approach, so I opted for integrating using a Struts plugin. The problem I encountered with the first approach was that I couldn't get the Scheduler to stop running after I stopped my servlet container (Tomcat 5.0.x). I suspect it may have been user error. If not, the problem could well have been resolved with the later releases.

The plugin approach was quite simple. Add a line like this to your struts-config.xml file:

<plug-in className="com.mycompany.plugin.QuartzPlugin"/>

Then create the QuartzPlugin class. It may look something like this:

public class QuartzPlugin implements PlugIn {
private String MY_JOB_GROUP = "MyJobGroup";
Scheduler sched = null;

public void init(ActionServlet servlet, ModuleConfig moduleConfig) throws ServletException {
log.info("Quartz starting");

try {
SchedulerFactory schedFact = new org.quartz.impl.StdSchedulerFactory();
sched = schedFact.getScheduler();
sched.start();

// Register job listeners with the scheduler
sched.addJobListener(new EmailWhenJobCompletesListener());

// Create the MyJob
JobDetail jobDetail = new JobDetail("MyJob", MY_JOB_GROUP, MyJob.class);

// Auto delete job from scheduler when no active triggers are associated with it.
jobDetail.setDurability(false);

// Do not persist between restarts of scheduler.
jobDetail.setVolatility(true);

// Pass stuff to the job with the JobDataMap.
String basePath = servlet.getServletContext().getRealPath("/");
jobDetail.getJobDataMap().put("basePath", basePath);
jobDetail.getJobDataMap().put("emailRecipients", MyConstants.JOB_STATUS_EMAIL_ADDRESS);
jobDetail.getJobDataMap().put("jobName", "My Job");

// Assign the EmailWhenJobCompletesListener to listen to this job and
// send an email when it is done.
jobDetail.addJobListener(EmailWhenJobCompletesListener.LISTENER_NAME);

// This is a trigger that will fire every night at 2:00am.
String cronExpression = "0 0 2 * * ?";
CronTrigger trigger = new CronTrigger("MyTrigger",MY_JOB_GROUP, "MyJob", MY_JOB_GROUP, cronExpression);

try {
// Schedule the job with the Scheduler.
sched.scheduleJob(jobDetail, trigger);
log.info("Scheduled the MyJob - " + cronExpression);
} catch (SchedulerException e) {
log.info("Quartz Scheduler failed to schedule the My Job: " + e.toString());
throw new ServletException(e);
}

} catch (Exception e) {
log.info("Quartz Scheduler failed to initialize: " + e.toString());
throw new ServletException(e); }
log.debug("Quartz started");
}

public void destroy() {
log.info("Quartz stopping");

try {
sched.shutdown();
} catch (SchedulerException ex) {
ex.printStackTrace();
}

sched = null;
}
}

And finally, create the job class, i.e., the class that will do what you want the job to do.

public class MyJob implements Job {
public MyJob() { }
public void execute(JobExecutionContext context) throws JobExecutionException {
JobDataMap dataMap = context.getJobDetail().getJobDataMap();
// Get a variable I passed to the job.
String basePath = dataMap.getString("basePath");
// Do stuff...

CashFlow Management Goes Live

After 7 months of development, the CashFlow Management project I have been working on with Ideal Financial Solutions has made it into production! There are, of course, a number of bugs that need to be worked out and a list of enhancements that will consume another 6 months of my nights and weekends, but it feels good to have something out in the market that is useable.

It has been quite a unique project to say the least. Ideal has a system they had created over the years and at the core was a monster of an Excel spreadsheet. I was asked to mimick the logic of the spreadsheet. Another unique aspect of the project was that the people involved in the project are separated geographically, so most meetings and communications were by phone or email. To make a long story short, everyone involved learned a lot about how to develop software and manage expectations.

The CashFlow Management system is one which helps people get out of debt and build wealth. A typical customer would provide Ideal, or one of its marketing call centers, with all of his financial data, including salary, monthly savings, and all debts and mortgages. This data is captured using the web-based application that my development team and I created. The data is then analyzed by the Ideal financial experts using tools provided in our web-based app. A proposal is created for the customer, which may include refinancing, credit card workout or settlement, adjustment of W2 withholdings, etc. The customer can then login to the web-based application and review it. If he chooses, he can sign-up for the program. His responsibility is simply to make sure that on a given day of the month, he has a certain amount of money in his checking account. This amount is the same amount, or less, than he is currently paying toward his debts. Ideal pays his debts off for him in the most effective way. It's not uncommon to see a 20+ year debt-free date cut to under 10 years.

iTracker - Open source bug tracker

I am working on a Debt Reduction/Wealth Builder software project for Ideal Financial Solutions where the team is dispersed geopgraphically. We have two developers, two testers, and some project manager types in four locations. We made it through the initial development phase with weekly phone calls, emails, and a few in-person meetings, but when we hit the testing phase, we had some big problems.

We started out by tracking the bugs in a couple of Word documents that we would email back and forth. The developers would add their comments on the issues and email the docs to the testers and vice versa. This got unwieldy after a couple of weeks, not to mention the attachments of the emails kept getting larger and larger.

I decided to look for an open-source (free) bug tracking tool and found iTracker. We've only been using it a couple of days now, but what a difference it has and will make for us.

I think my biggest gripe is the lack of detailed documentation, but that is to be expected with most open-source software. I originally wanted to use a MS SQL Server database as the backend, but quickly discovered I would have to spend a weekend pouring over JBoss installation/configuration documents in order to understand exactly what the limited documentation was asking me to do. (I only have Tomcat experience.)

Thank goodness that iTracker ships with an "Express Install". The Express Install uses HSQL, not sure if that is a JBoss specific persistence mechanism, or another open-source package. Using the Express Install I was up and running in less than an hour. I still need to figure out why the email notification isn't working. It appears to be an SMTP server problem. I contacted my ISP and they gave me the SMTP server info, but email notification is still not working.

I remember seeing some posts at the iTracker forum regarding troubleshooting email notifications. I'll have to do a little research and get it going.

Update - Email notification now working
Since I was running outside of my ISP's network, I had to provide authentication information. After I specified that in the itrackerApplication.properties file I still had problems. The last thing I had to do was provide valid email addresses in itrackerApplication.properties. I had some bogus email address for the from and reply. After changing them to real email addresses, email notifications started working.

jTDS - The fastest JDBC driver for MSSQL

jTDS is an open source 100% pure Java (type 4) JDBC driver for Microsoft SQL Server. It claims to be the fastest JDBC driver around and backs the claim with some benchmarks. All you have to do is drop a jar in the classpath, change your jdbc driver settings and url, and you're good to go, or so they say.

I dropped the jar file, modified my jdbc connection info as specified in their FAQs, but now my app won't start. I suspect it's my fault somehow, but it is a bit perplexing.

My app has some startup code that loops through all the resources in the environment context "java:comp/env". For some reason, my jTDS datasource is not found in the environment. If I replace the Driver and URL with the original Microsoft driver values, it does show up in the environment.

Here is my resource entry in Tomcat server.xml file:

<resource type="net.sourceforge.jtds.jdbcx.JtdsDataSource" auth="Container" name="mypool">
<resourceparams name="mypool">
<parameter>
<name>factory</name>
<value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
</parameter>
<parameter>
<name>maxActive</name>
<value>50</value>
</parameter>
<parameter>
<name>maxIdle</name>
<value>25</value>
</parameter>
<parameter>
<name>maxWait</name>
<value>10000</value>
</parameter>
<parameter>
<name>username</name>
<value>the_user</value>
</parameter>
<parameter>
<name>password</name>
<value>the_password</value>
</parameter>
<parameter>
<name>driverClassName</name>
<value>net.sourceforge.jtds.jdbc.Driver</value>
</parameter>
<parameter>
<name>url</name>
<value>jdbc:jtds:sqlserver://server1:52500/dev_2.2;
user=the_user;password=the_password</value>
</parameter>
<parameter>
<name>validationQuery</name>
<value>SELECT 1</value>
</parameter>
<parameter>
<name>logAbandoned</name>
<value>true</value>
</parameter>
<parameter>
<name>removeAbandoned</name>
<value>true</value>
</parameter>
<parameter>
<name>removeAbandonedTimeout</name>
<value>300</value>
</parameter>
</resourceparams>

And the code that fails is:

Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
Object obj = envCtx.lookup("mypool");

After the last line of code above, obj is null. For some reason, the datasource is not getting into the environment context.

I posted the above problem on a jTDS forum (http://sourceforge.net/forum/forum.php?thread_id=1226037&forum_id=104389) and received a suggestion that fixed the problem.

I needed to change "net.sourceforge.jtds.jdbcx.JtdsDataSource" to "javax.sql.DataSource" in the <resource> element. I think I misunderstood the jTDS documentation. There was a table that had two columns, one column had "javax.sql.DataSource" and the other had "net.sourceforge.jtds.jdbcx.JtdsDataSource". I thought it was a simple "search and replace" job, but not so.

Anyway, it's up and running now!

StrutsWS (Web Service)

StrutsWS is a relativley new open-source project on Sourceforge http://sourceforge.net/projects/strutsws/. Frank Zametti, the author, has mentioned it various times recently on the struts user list. He mentioned today that he had just released a new MAJOR update, so I decided to take a look.

Here is Frank's abstract of the project:

This project is an attempt to create a mechanism in Struts whereby existing business logic components of a Struts-based application can be exposed as rudimentary Web Services without the need to change any code. To do this, I have created a custom RequestProcessor class. This class recognizes a Web Service request, that is, a SOAP message over HTTP, performs some pre-processing on the request, and passes it along to the targeted Action in a form it will understand. It then generates a SOAP-based XML response to the request in place of the usual view components of the existing application.

I read through the overview document and it sounded like an interesting project. It's main strength, in my opinion, is that you can use your existing Actions and ActionForms. There is no need to rewrite business objects to make them web services. Unfortunately, if an application makes heavy use of Session data, like session-scoped ActionForms or for handling security, then the StrutsWS library is of little value, since you now cannot use existing Actions and ActionForms. To quote Frank, "And of course, anything requiring session will NOT work, since no session would exist for a one-off request like these. "

I need to investigate Apache Axis a bit more and see how complicated it is use install and use.

Trying out formdef

I decided to try out Hubert Rabago's formdef Struts plugin. After working through some silly mistakes, like getting the correct version of the formdef library for Struts 1.1, I think I finally got it working.

The framework we originally put together did not address the issue of business objects and Struts ActionForms very well. We naively just added as a member variable to each ActionForm (ValidatorForm) a data transfer object (DTO). These DTOs were the same ones used by the data access layer, so the data types of the variables in these "beans" were data types that the database expected. This is not ideal, as web forms should typically only deal with strings and maybe booleans.

As we got further into implementation and development using this paradigm, we discovered that we had to worry about data type conversions for displaying the data on a web form. It got a little cumbersome as a developer.

I stumbled across the formdef Struts plugin and it sounded like a good option. It introduced us to Dyna forms, which we had read about, but never investigated much. They're actually quite nice and save the developer the time of having to create ActionForm classes for each form. formdef takes it one step further by creating your dyna ActionForms for you based on a DTO (bean) that you specify, so the ActionForms match the business objects.

formdef also provides built in conversion methods to move data from the DTO to the form and vice versa. It is a very flexible library, providing many hooks and overrideables, so it should serve us well.

There was a thread on the Struts user list Nov 2004 that talked about this subject a little. Here is the URL to the original post. You can step through the response using the "next in thread" option. http://marc.theaimsgroup.com/?l=struts-user&m=110001898604664&w=2

[UPDATE 27Aug05]
The past couple of days I have tried to use formdef in a real web application and have run into a few problems. First off, I don't think formdef supports multi-select lists, i.e., <select multiple="multiple">. Second, I'm not sure how to implement a one-to-many relationship using formdef. Let me explain...

I use iBATIS for the DAO and db connection functionality. One feature iBATIS provides is that I can populate a DTO object from a query and the DTO can have a List property that stores the "many" side of the relationship. iBATIS will populate that List property automatically for me with all the related db rows.

I have not had success in creating the DynaValidatorForm when I have a List object in my DTO. formdef seems to choke on it, because when I remove it, everything is fine. Maybe there is a better way to handle one-to-many relationships when using formdef, not sure ??

Initial Post

I decided to start my own blog! I'll be blogging stuff I learn and discover as I work on a variety of computer consulting/contracting jobs. It'll be nice to have a repository for such findings, but really, it just kinda makes me feel cool that I am a blogger now :)