Sunday, September 5, 2010

ASP MVC 2 – NUnit based unit test project template For Visual Studio 2008

Piotr Kwapin wrote an article about creating a NUnit based unit test project template for ASP MVC2 for Visual Studio 2010. I am still using Visual Studio 2008. This was an interesting learning experience. I never previously delved into project templates.

Piotr Kwapin mentioned this would work with Visual Studio 2008 but did not elaborate. I was able to get it working after some rather scary moments.

Download the Template

The first step is to download the template updated for Visual Studio 2008. The template is a zip file containing the following.

  • Controllers
    • AccountControllerTest.cs
    • HomeControllerTest.cs
  • Properties
    • AssemblyInfo.cs
  • App.config
  • MvcApplicationTest.csproj
  • MvcWebApplicationNUnitBasedTestProjectTemplate.cs.vstemplate

Put the template file in the template directory. On a PC running a 64-bit operating system, this directory is located here. Do not unzip or extract files from the zip.

c:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\ProjectTemplates\CSharp\Test\

Download the Registration File

The next step is downloading the updated registration file. Extract the registration file and execute it. Windows will issues several warning and confirmations about running the file.

Running Devenv Setup

Piotr Kwapin did not document the last step as completely. You need to open up a command line as administrator that has the path setup to run devenv.exe. On Windows 7, select Start, All Programs, Visual Studio 2008, and then Visual Studio Tools.

 

Right click on Visual Studio 2008 Command Prompt and select Run as administrator. Enter the following command.

devenv /setup

 

The first time I did that, Visual Studio 2008 forgot about all its templates. I suggest running the command a couple of times. I ran the command around a dozen times as I made minor changes to the files to get them to work with Visual Studio 2008. I documented the changes I made to the files below.

Changes to the VSTemplate File

I changed the version number for Microsoft Visual Studio from 10.0.0.0 to 9.0.0.0. Below is the updated line.

<Assembly>Microsoft.VisualStudio.Web.Mvc.2.0, Version=9.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35</Assembly>

Changes to the CSProj file

I changed the tools version from 4.0 to 3.5. Below is the updated line.

<Project ToolsVersion="3.5" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Changes to the Registration File

I made two changes to the registration file. The first is a change to the name of the template. I compressed the template files into NUnitTestVS9.zip. The second change is the change of the HKEY to Visual Studio 2008. The two lines below show the changes.

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\9.0\MVC2\TestProjectTemplates\NUnit\C#]

"Template"="NUnitTestVS9.zip"

A New ASP.NET MVC 2 Web Application

Creating a new ASP.Net MVC 2 Web Application in Visual Studio 2008 creates both the project and allows creating a NUnit test project. Below is an example in Solution Explorer.

 

Wednesday, August 11, 2010

Hudson, SmartFoxServer, Plink, and Running Command as Local System Account

SmartFoxServer is a platform for developing massively multi-user applications. SmartFoxServer is a Java application and supports calling extensions in Java, Action Script, and Python. As a Java application, SmartFoxServer runs well on Windows or Linux.

Ryzing tested running SmartFoxServer on Windows and Linux. We ended up using an Amazon EC2 instance running Ubuntu Linux to host the SmartFoxServer. SmartFoxServer interprets the Action Script extensions. You cannot compile the extensions. SmartFoxServer caches all the Action Script extensions. The server will not reload the extensions if they change. SmartFox developed an admin Flash client application to manage the SmartFoxServer. You can use the Flash admin client to force extensions to reload.

I designed the Hudson build process at Ryzing to push the SmartFoxServer extensions to the Linux server. Fortunately, SmartFox implemented a set of command line tools in addition to the Flash client. The process of pushing the extensions to the server is a simple task in Hudson.

  1. Extract the source from SVN
  2. Rename some files to match the destination server
  3. Push the sources to the server using PSCP
  4. Reset the SmartFoxServer

Pscp is an implementation of scp or secure copy that runs on Windows. Pscp is one of the tools included with Putty. The process should be simple. However, there is a problem. The first time Putty, Pscp, or Plink connects to a server, they cache the host key in the registry. You get something like the following.

The server's host key is not cached in the registry. You
have no guarantee that the server is the computer you
think it is.
The server's rsa2 key fingerprint is:
ssh-rsa 2048 hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh:hh
If you trust this host, enter "y" to add the key to
PuTTY's cache and carry on connecting.
If you want to carry on connecting just once, without
adding the key to the cache, enter "n".
If you do not trust this host, press Return to abandon the
connection.
Store key in cache? (y/n)

Putty, Pscp, and Plink support a batch option. With the batch option, if the host key is not in the registry cache Putty, Pscp, and Plink automatically exit. The problem is the registry cache is user specific. This causes a problem with Hudson installed as a service running on Windows. When installed as a service, Hudson runs as the Local System Account. Running Pscp as a task in a Hudson project will lock up waiting for the user to press "Y" or "N". If you add the option to run as batch, the command will always fail.

How do you install the host key in the registry for the Local System Account? I am sure there are other solutions. The one I found was to open a command prompt as the Local System Account. Unfortunately, this is not as simple as it sounds. I finally found the solution here: http://johnnycoder.com/blog/2008/11/10/run-cmdexe-as-local-system-account/.

The solution is to use psexec.exe, which is part of PsTools from SysInternals. Microsoft acquired Sysinternals in July 2006. The following opens a command prompt as the Local System Account.

psexec -i -s cmd.exe

After the command window opens, attempt to connect via Pscp or Plink. Answer "Y" to "Store key in cache?"

Now Hudson can copy the files from SVN to the Linux server and restart SmartFoxServer.

Tuesday, August 10, 2010

Using Hudson for Unusual Tasks

Martin Fowler wrote, "Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly." (http://martinfowler.com/articles/continuousIntegration.html)

Hudson is a continuous integration solution. "In a nutshell, Hudson provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build." (http://wiki.hudson-ci.org/display/HUDSON/Meet+Hudson)

I found somewhat unusual uses for Hudson.

Ryzing follows a standard development setup with a set of servers (or chain) for each stage of development: Alpha, Quality Assurance (or QA), and Production. We chose the Amazon Cloud to host the servers. Three chains mean three separate databases running on three separate database servers. I firmly believe in automating routine tasks. A daily backup is definitely a routine task. My solution to automating the backups was to use our Hudson server.

The Ryzing applications use PostgreSQL as the database server. I installed the full PostgreSQL tool set on the Windows 2008 server dedicated to running Hudson (http://www.postgresql.org/download/windows). I made sure that both the Windows firewall and the Amazon security zones allowed the build server to talk to the database servers in the Alpha, QA, and Production chains.

The project in Hudson is simple. The build process uses a Windows batch command. Below are the commands I used.

SET PGPASSWORD=PASSWORD
"C:\Program Files\PostgreSQL\8.4\bin\pg_dump.exe" --no-password --file=DATABASENAME.backup --format=c --host=HOSTNAME --port=HOSTPORT -U USERNAME DATABASENAME

You will need to set the appropriate password, database name, host name, user name, and host port. I checked a Post-Build action to Archive the Artifacts telling Hudson to archive DATABASENAME.backup. I selected the build trigger of building periodically. Each morning the build server runs the backup on each database.

Hudson provides a convenient place to store the database backups along with all our software builds.

Monday, August 9, 2010

The Joys of MeidaWiki and VMware

Wikis are a useful tool. At my current position, I added a lot of technical documentation to the Wiki included in Trac. I wanted a wiki for my personal use.

I program in a variety of languages including C# and Java. Most of my recent work is in C#. I looked for a Wiki in C#. I found two.

  1. FlexWiki
  2. MiniWiki

I know there are others. Unfortunately, FlexWiki is abandoned. The source code is available. MiniWiki is just that a minimal Wiki. MiniWiki is interesting because it uses the ASP.Net MVC (version 1).

Both include unit tests. I am a bit of a fanatic about unit tests. In Working Effectively with Legacy Code Michael Feathers wrote, "Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse."

MiniWiki uses Moq. FlexWiki uses NUnit. Interesting neither include any logging framework.

What I would like to do, if I ever find the time, is take the Wiki engine from FlexWiki and marry it with the MVC framework using MiniWiki as a start. That is a good long-term project.

In the short term, I need a personal wiki. The solution is MediaWiki and VMware. rPath created a MediaWiki appliance http://wiki.rpath.com/wiki/Appliance:MediaWiki_Appliance

With the appliance I was up and running in minutes.

Last thing to do was to modify the .vmx file in the appliance to use a static MAC address. Comment out the following lines.

ethernet0.addressType = "generated"
ethernet0.generatedAddress = "00:0c:29:43:85:c8"
ethernet0.generatedAddressOffset = "0"

Then add something like the following.

ethernet0.addressType = "static"
ethernet0.address = "00:50:56:00:00:04"

The MAC address you select depends on your network. I want a static MAC address to allow my Netgear router to assign the IP address. This gives the VM a fixed IP address inside my firewall.

Sunday, August 8, 2010

Fixing the Certificate Error Problem with VMware Server

I use VMware server for many thing. For example, I own an older version of QuickBooks to bill for consulting projects. I use QuickBooks once a month to create invoices. For me, it is not worth the cost of upgrading to a new version of QuickBooks. I have a VM of Windows XP that runs my copy of QuickBooks.

One problem with VMware Server is Internet Explorer needs the certificate for the VMware Infrastructure Web Access program. Otherwise, you get a certificate error. VMware works fine if you ignore the error but it is easy to fix, once you find out how.

The biggest challenge is finding the certificate. I run Windows 7. The find for Windows is useless. It rarely finds any file I am looking for. A slower, but more reliable solution is the Gnu FindUtils for Windows. You can download a copy here:

http://gnuwin32.sourceforge.net/packages/findutils.htm

Another useful utility is Gnu Grep for Windows. You can download a copy here:

http://gnuwin32.sourceforge.net/packages/grep.htm

On Windows 6 (64 bit) the installation packages for FindUtils and Grep install the executables in C:\Program Files (x86)\GnuWin32\bin

The standard Windows command find is useless; but makes it impossible to execute the Gnu find. I rename Gnu's find.exe to gnufind.exe. I then add the location to the environment path variable. That allows running grep or gnufind from any command window.

Gnufind located the certificate use by VMware named rui.cert here C:\Program Files (x86)\VMware\VMware Server\SSL.

Once you know find the certificate, installing it is simple.

  • Open up Windows Explore and browse to the location of the certificate.
  • Double click on rui.cert

The following dialog will display.

Clicking on the Install Certificate brings up the wizard.

Select Place all certificates in the following store. Then click the Browse button.

Select Trusted Root Certificate Authorities.

Click OK

Click Next

Click Finish.

Close the browser. Now when you open up VMware server the cerficate problem is solved.

Wednesday, November 4, 2009

Log Timmer

One common challenge in developing software is tuning performance. The application is running slow but you are not sure what is causing the problem. An approach to the debugging the problem is to use logging. This works, but you end up doing a lot of time arithmetic to determine how long it took some action to complete. Logging also adds many lines of code to the application muddling up the implementation and making it harder to read and understand. The solution is a class designed to time and log an event. I give credit to Scott Plichta for designing the architecture for the Log Timer.

Creating a log in C++ is easy. The constructor starts the timer and the destructor calculates the elapsed time and logs the results. This is a problem in Java. In Java, destructors are may not be called until the garbage collector runs. C# also uses garbage collection, but provides the IDisposable interface. IDisposable tells the garbage collector to dispose of the object as soon as possible. That means the timings provided by a log timer class in C# are not quite as accurate as the timing in a C++ log timer class.

How do you use the log timer class work in C#? The simplest method is just to place a "using" statement creating the log timer around the code you want to time:

using (new LogTimer.LogTimer())
{
// code you want to time
}

Call that is a little more complicated allows passing in the custom message:

using (new LogTimer.LogTimer("Deserialize time: "))
{
// code you want to time
}

The code above will always log. The LogTimer class has constructors that allow passing in the number of milliseconds for logging for each of the Log4Net levels. Below is an example passing in all the level thresholds:

using (new LogTimer.LogTimer("DownloadImage GetReponse time: ", 5000, 3000, 2000, 100, 0))
{
// code you want to time
}

There are several other constructors taking different number of thresholds.  Below is the implementation of the LogTimer class in C#.

using System;
using System.Collections.Generic;
using System.Text;

using log4net.Core;
using System.Diagnostics;

namespace LogTimer
{
// based on:
// http://www.eggheadcafe.com/articles/20050625.asp
// http://www.codeproject.com/KB/cs/idispose.aspx

public class LogTimer : IDisposable
{
/// <summary>
/// Create a logger for use in this class
/// </summary>
private static readonly log4net.ILog log = log4net.LogManager.GetLogger(typeof(LogTimer));

/// <summary>
/// Used by the destructor to avoid disposing twice
/// </summary>
private bool disposed = false;

/// <summary>
/// Default message
/// </summary>
private string msg = "LogTimer: ";

/// <summary>
/// Threashold in milliseconds before issuing a debug log
/// </summary>
private readonly long debugCount = 0;

/// <summary>
/// Threashold in milliseconds before issuing a info log
/// </summary>
private readonly long infoCount = 0;

/// <summary>
/// Threashold in milliseconds before issuing a warn log
/// </summary>
private readonly long warnCount = 0;

/// <summary>
/// Threashold in milliseconds before issuing a error log
/// </summary>
private readonly long errorCount = 0;

/// <summary>
/// Threashold in milliseconds before issuing a fatal log
/// </summary>
private readonly long fatalCount = 0;

/// <summary>
/// Stopwatch to determine how long between starting and stopping
/// </summary>
private Stopwatch sw = new Stopwatch();


/// <summary>
/// Simple constructor that generates a default message for the current log level
/// </summary>
public LogTimer()
{
sw.Start();
}

/// <summary>
/// Simple constructor that generates the messages for the current log level
/// </summary>
/// <param name="message">The message to log</param>
public LogTimer(string message) : base()
{
msg = message;
}

/// <summary>
/// Most comprehensive constructor. Allows setting all the properties.
/// </summary>
/// <param name="message">Message to log</param>
/// <param name="fatal">Threshold for fatal logs</param>
/// <param name="error">Threashold for error logs</param>
/// <param name="warn">Threashold for warn logs</param>
/// <param name="info">Threashold for info logs</param>
/// <param name="debug">Threadhold for debug logs</param>
public LogTimer(string message, long fatal, long error, long warn, long info, long debug) : base()
{
msg = message;
fatalCount = fatal;
errorCount = error;
warnCount = warn;
infoCount = info;
debugCount = debug;
}

/// <summary>
/// Construct allowing setting the message and threasholds for fatal, error, warn, and info.
/// </summary>
/// <param name="message">Message to log</param>
/// <param name="fatal">Threshold for fatal logs</param>
/// <param name="error">Threashold for error logs</param>
/// <param name="warn">Threashold for warn logs</param>
/// <param name="info">Threashold for info and debug logs</param>
public LogTimer(string message, long fatal, long error, long warn, long info) : base()
{
msg = message;
fatalCount = fatal;
errorCount = error;
warnCount = warn;
infoCount = info;
debugCount = info;
}

/// <summary>
/// Construct allowing setting the message and threasholds for
/// fatal, error, and warn.
/// </summary>
/// <param name="message">Message to log</param>
/// <param name="fatal">Threshold for fatal logs</param>
/// <param name="error">Threashold for error logs</param>
/// <param name="warn">Threashold for warn, info and debug logs</param>
public LogTimer(string message, long fatal, long error, long warn) : base()
{
msg = message;
fatalCount = fatal;
errorCount = error;
warnCount = warn;
infoCount = warn;
debugCount = warn;
}

/// <summary>
/// Construct allowing setting the message and threasholds for
/// fatal and error
/// </summary>
/// <param name="message">Message to log</param>
/// <param name="fatal">Threshold for fatal logs</param>
/// <param name="error">Threashold for error, warn, info and debug logs</param>
public LogTimer(string message, long fatal, long error) : base()
{
msg = message;
fatalCount = fatal;
errorCount = error;
warnCount = error;
infoCount = error;
debugCount = error;
}

/// <summary>
/// Construct allowing setting the message and threashold
/// </summary>
/// <param name="message">Message to log</param>
/// <param name="fatal">Threshold for fatal, error, warn, info,
/// and debug logs</param>
public LogTimer(string message, long fatal) : base()
{
msg = message;
fatalCount = fatal;
errorCount = fatal;
warnCount = fatal;
infoCount = fatal;
debugCount = fatal;
}

/// <summary>
/// Actual destructor
/// </summary>
~LogTimer()
{
Dispose(false);
}

/// <summary>
/// Dispose method
/// </summary>
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}

/// <summary>
/// Dispos method
/// </summary>
/// <param name="disposing">currently disposing</param>
protected virtual void Dispose(bool disposing)
{
if (!this.disposed)
{
if (disposing)
{
sw.Stop();
long duration = sw.ElapsedMilliseconds;

// log with the highest enabled log level
// a log level is enable when IsLevelEnable and leveCount > 0
if (log.IsFatalEnabled && duration >= fatalCount)
{
log.Fatal(msg + duration.ToString());
}
else if (log.IsErrorEnabled && duration >= errorCount)
{
log.Error(msg + duration.ToString());
}
else if (log.IsWarnEnabled && duration >= warnCount)
{
log.Warn(msg + duration.ToString());
}
else if (log.IsInfoEnabled && duration >= infoCount)
{
log.Info(msg + duration.ToString());
}
else if (log.IsDebugEnabled && duration >= debugCount)
{
log.Debug(msg + duration.ToString());
}
}
}
disposed = true;
}

}
}

Monday, August 24, 2009

The Lost Weekend: IIS 7, PHP, and Zend

My plan for this weekend was to implement a new version of one of my web sites. I wanted to move from an old complicated build process to one based on using the Zend conventional modular layout. I estimated I could complete the task over the weekend. I was wrong.

Microsoft made a significant change by allowing IIS to run on the home versions of Vista. I could install IIS 7 on my new Vista 64 bit PC. I thought it would be a fun learning experience to use IIS 7 as a test bed while I implemented the changes. I was wrong.

Microsoft packaged up PHP and created an install using their new Web Platform Installer. This sounded like a quick way to get PHP up and running. If you want to get PHP up and running quickly on IIS 7, the Web Platform Installer will do that. However, if you want to use PEAR, the Web Platform Installer is a dead end. I have a problem with PEAR. I was unable to find any instructions on how to install PEAR manually. Everywhere I looked the documentation always said to run the "go-pear" script or batch file. Microsoft decided not to include that little file in their installation of PHP via the Web Platform Installer. When you install PHP via the Web Platform Installer on Vista 64 PHP is installed in the "\Program Files (x86)\php" directory. The installer then sets the include path to ".;c:\php5\pear" even though there is no php5 directory created. If you want PEAR and PHP on IIS, you want to avoid using the Web Platform Installer. Instead, use the "installer version" found at "windows.php.net". When you install, make sure you change the installation directory to "\php5". I also recommend you install PEAR, which is an option with the "installer version". I thought the Web Platform Installer quickly set up PHP and PEAR. I was wrong.

When you install IIS 7, it does not include a URL Rewrite Module. Only two things were quick and easy this weekend. The first was installing the IIS URL Rewrite Module and importing the rewrite rules from Apache. The second was downloading and installing the Zend Framework. The Zend Framework is a .Zip file you download, and decompress to anywhere you choose.

I used the Zend Framework before. I never started a project using MVC from scratch. I wanted to use the conventional modular layout to support three levels. There were tutorials that showed how to set this up. It looked very simple to me. I was wrong.

In previous entries, I mentioned "Practical Web 2.0 Applications with PHP" by Quentin Zervaas. Apress published the book in December of 2008. It is already out of date. About the time the book was published version 1.8 of the framework was released which deprecated the use of Zend_Loader's autoloader. I thought I could use the tutorial in the book by Zervaas to build a site from scratch. I was wrong.

I set up my site with a single controller. I kept getting the error "Invalid controller specified". I spent hours checking my directory structure. I googled the error message. Most web sites suggested that there was a problem with the file or directory casing. I am using IIS 7 on Vista. Files and directory casing should not be an issue. I thought I had some sort of typo in some file. I was wrong.

Finally, after much pain and cursing I found the problem. The problem is auto-rendering. The Zend framework automatically goes hunting for the template with auto-rendering enabled. Most of the sample code has a hidden assumption that auto-rendering is off. This is no longer true with the default configuration. I was getting the error "Invalid controller specified". The problem was not with the controller but with the view. Here is a situation where the MVC separation in Zend is poorly constructed. Zend generates an inappropriate error message.

Overall, I lost a weekend getting IIS 7, PHP, and Zend working. I did not finish redesigning my website. I still have a lot of work ahead. I was not expecting a major learning experience this weekend. I was wrong.