Sunday, June 7, 2009

Running Mercurial on Windows

Mercurial, if you haven't played with it yet, is one of the easiest, fastest source control mechanisms to hit Windows in recent years. If you have heard of Mercurial (or Hg, as some folks will type it, the symbol on the periodic table of elements for mercury), you may think I misspoke with that "Windows" bit, since only those long-haired open source hippie Linux people use that freebie distributed version control. Not true! As a matter of fact, for your own personal projects, Hg is probably one of the easiest, fasted version control mechanisms you can use for local-only development.


Getting started

The basic reference material you will need for Hg is available in the Mercurial wiki at http://www.selenic.com/mercurial/wiki/. Just so you don't have to read it, I'll continue below.

On your local Windows box, you will want to install TortoiseHg. If anybody has ever used the fine TortoiseSVN on Windows, you will be immediately familiar with the direct Windows Explorer integration with source control. Some of the more recent versions of TortoiseHg (0.7.6 is the latest as of this writing) include a full installation of the command-line version of Mercurial, saving Windows uses the hassle of finding and running another installer. You will eventually come up with a need for the command-line version of Hg, and you will want to become familiar with its simple commands anyway.

For Visual Studio users (Standard edition and above, sorry Express edition users. Out in the cold as usual.), you can install VisualHg to get that sweet, sweet development environment integration that the users of Visual Source Safe and TFS currently enjoy in their development experience. Once installed, you can enable the source control integration within Visual Studio by going to Tools->Options, then choosing "Source Control" in the tree view, and changing the "Current source control plug-in:" combo box.


As a side note, if you're still using VSS...please stop. SVN and Mercurial are, at the very least free, and are far less likely to corrupt your source repository. Also, if you aren't paying for each and every seat license of VSS (even if you put it on a server and share it), you're stealing from MS at the rate of about $390 a seat. Don't place your business and your code at risk by using VSS.


Beyond local development


Despite what some other ill-informed (yet admittedly old) blogs have implied, Mercurial can be hosted in IIS on Windows. Nobody's set out step-by-step instructions all in one place, but that is about to change.


For all versions of Windows


  1. Install Mercurial 1.2.1 on the server. The version that comes with TortoiseHg is not sufficient, it will not drive the CGI application.

  2. Install Python 2.5.4 on your server. Mercurial 1.2.1 components are compiled against that version of Python. Any other version of Python will cause a "Bad Magic Number" error on the server.

  3. Create a directory somewhere to hold the Mercurial source tree. Open a cmd window and navigate to whichever directory you want this source tree to be contained within, then type in
    hg clone http://selenic.com/repo/hg
    (robbed from Stack Overflow). This command will create a directory named hg, then download all the code in the Mercurial source repository into the hg directory. What we're after here is is the hgwebdir.cgi file, which is...a Common Gateway Interface app. Duh. And I'll bet you thought that cgi was dead.

  4. Create a directory on the file system (separate from any other web app files you have on your web server) to map in as a Virtual Directory in IIS. I used C:\inetpub\hgcgi\. Make sure this directory has read access in NTFS for the service account IIS (or your IIS app pool for Vista, XP x86-64, Server 03 and Server 08) is running under.

  5. Copy the hgwebdir.cgi file into your virtual-directory-to-be.

  6. You also want to extract the contents of the Library.zip file contained in your Mercurial installation directory to your virtual-directory-to-be. If you installed TortoiseHg 0.7.6, you will find this within the TortoiseHg installation directory.

  7. Copy the entire "templates" directory from the Mercurial installation directory to a "templates" directory within the virtual-directory-to-be.

  8. Create a file named hgweb.config. This is absolutely nothing like an Asp.Net web.config. Its more like an ini file from the days of old. And you have to actually add text to it. You want something that looks like this:

    [paths]
    MySourceCode = C:\Repositories\**
    [web]
    style = monoblue

    The [paths] segment is in the form = local path. You can list out repository locations manually, or you can do as above, and have it take all of the repos in a location. And yes, the ** is correct. Most of the other documentation around the web is unix-style fs specific, and suggest using the [collections] section as opposed to the [paths] section, but don't be surprised when it doesn't work on Windows. I never had any luck with using the [collections] section.
    And yeah, you want to change the style. Because you can.

  9. Once you get your repos to show up in the Mercurial cgi script, you can go into the repo's .hg directory, find the hgrc file therein and configure it. The syntax is the same as hgweb.config, and it allows you to set settings for the repository itself. Such as the style. Because you can. Details on the config file are here.


That's everything you must do that is common to all versions of Windows.


IIS 7 (Windows Vista, Server 08)

Just like everything else that is related to Vista, Microsoft changed the whole UI for the IIS Manager. This is good, because IIS Manager (just like the Component Services snap-in) in Server 2003 is a fussy, buggy, expletive-inducing wreck. IIS 7 does give you the ability to install Asp.Net "filters" (called a "Managed Handler" in IIS Parlance, called an HttpHandler in Asp.Net parlance) directly into IIS. Kinda cool really, since writing ISAPI filters requires that icky native code that seems to bamboozle so many people. Maybe someone will write a nice Managed Handler for Mercurial in the future, but for now, we have to use CGI.

  1. Install IIS. In Vista, its Control Panel->Programs and Features in "Classic View" or Control Panel->Programs in the new Vista view. Click on "Turn Windows features on or off." Expand "Internet Information Services", then select "Web Managment Tools" to install IIS Management Console. Select "World Wide Web Services", then expand "World Wide Web Services", then expand "Application Development Features" and make sure "CGI" is selected.

  2. Add your virtual directory for Mercurial. That's the one that has hgwebdir.cgi in it. Right click a web site (I just mapped it into the "Default Web Site", but any will do.), and choose "Add application" You could "Add virtual directory" and then upgrade the virtual directory to an application, but the virtual directory mapping has to be an "application." In the dialog, I set the alias to hg and the Physical path to "C:\inetpub\hgcgi". There is a nice "Test settings..." button here that will help you determine if your application directory is configured property. I never did get my authorization warning to go away, even though the CGI script would run.

  3. Map in the Python interpreter as a script engine. Click on your new application/virtual directory. In the center pane, double-click "Handler Mappings." Normally, if you wanted to execute Python scripts on the server, you would map the interpreter to *.py. But we're running a cgi app through the python interpreter. Click on "Add Script Map..." and put in *.cgi for the Request Path. Put
    C:\Python25\python.exe -u %s %s
    in for Executable. Give your mapping a name. I used Python 2.5. The %s %s mess above are substitution symbols (like printf(), or string.Format()) for arguments passed to the python interpreter that tell it what script it is executing on behalf of IIS. After that, click the "Request Restrictions..." button, and choose the "Access" tab. Pick the "Execute" option, so you can "Execute" the cgi application. Click OK to save. A dialog will pop up that says "Do you want to enable this ISAPI application?", which is a bit bizarre since you are enabling a CGI application, but you should say Yes.

  4. Enable the script engine for your application/virtual directory. Right click your new "Python 2.5" handler mapping, and select "Edit feature permissions...", and make sure all of the checkboxes there are checked.


Windows Server 2003/XP


  1. Buy Server 2008. I hear that if you get it on subscription, you get free upgrades. If you can't talk your boss into paying for the next scheduled maintenance on Balmer's V12 Vantage (and if he doesn't really have one, he probably should), make sure IIS is installed on the Server. Control Panel->Add or Remove Programs, then Add/Remove Windows Components. Select Application Server, and you should be set.

  2. Add your virtual directory for Mercurial. That's the one that has hgwebdir.cgi in it. Right click a web site (I just mapped it into the "Default Web Site", but any will do), and choose New->Virtual Directory. In the dialog, I set the alias to hg and the Physical path to "C:\inetpub\hgcgi". By default, in Server 03, virtual directories are mapped in as "Web Applications," but if it isn't set up that way, you can right click the virtual directory, and click the "Home Directory" tab. On the bottom of that tab, in the "Application settings" section, there should be a button labeled "Add." Once you click it, the virtual directory becomes a "Web Application." Also, while you're there, make sure the "Execute permissions:" drop down says "Execute."

  3. Map python in as the cgi script handler. Do this by right clicking the virtual directory, then selecting the "Home Directory" tab. In the "Application settings" section, click the "Configuration" button. On the "Mappings" tab, click the "Add..." button. For "Executable", put in the following:
    C:\Python25\python.exe -u "%s %s"
    And make sure you put the double quotes in exactly as shown. This is because IIS Manager is and won't take it without them. In "Extension", put ".cgi". Clear the checkbox that says "Verify that file exists", set the checkbox that says "Script engine."

  4. Enable Python as Web Service Extension (Server 03/XP x64). In IIS Manager, right-click the Web Service Extension to "Add a new Web service extension...". You can name the extension anything you want, I chose "Python 2.5". Click the "Add..." button. In "Path to file", put C:\Python25\python.exe -u "%s %s". It must exactly match the ".cgi" mapping on your virtual directory, or it won't work. Check "Set extension status to Allowed".


Point your browser to http://server/HgVDir/hgwebdir.cgi/ and enjoy. You should now be able to use Mercurial against that virtual directory. To secure your source control repository, you should set up SSL on your server, and mark your hg virtual directory to require SSL. Then set the virtual directory to use Basic security, as this is what the hg client tools support. Basic-over-SSL is pretty secure, since the SSL encrypted connection is established well before the Basic security credential exchange takes place.

To create new repos on the server, you'll want to clone them to the server from your workstation machine. The easiest way to do this is to hg serve from your workstation, then log onto the server and pull your new repo into the right place on the server from your workstation. After that, go into .hg in your server-side repo and configure your hgrc file. Once that is complete, you can synchronize your server repo with your workstation repo any time you wish.

And just in case you're wondering, the -u switch on the python interpreter places the interpreter in "unbuffered" mode, so results are sent to IIS immediately with no delay. Supposedly, for CGI, it improves performance.

Give hg a shot. Its effective and I've had quite a bit of success with it. Maybe you'll like it, too.



32 comments:

  1. Can the repositories safely be hosted on a SAN or networked disc?

    And can the path be a network location such as: \\myserver\repositories or must the network drive be mapped as a regular drive I.e: F:\...?

    ReplyDelete
  2. I haven't had any experience with Hg and network drives, but I don't really think Hg will much mind. The problem you're going to have is with permissions accessing the drive share.

    I think that the CGI app will try to access the file share with IIS's service account creds (if you have anonymous access enabled), which by default will probably be either "Network Service" or IUSR_<whatever>, which don't usually have any real permissions on another machine.

    It will impersonate the caller if you have anonymous access turned off, or you have security set up on your drive share...and you should make sure your drive share is restricted to developers only, so the people in accounting don't accidentally break your repo.

    Also, I don't know if you will have trouble with it or not, but on my server, I have had issues with the pull command and the CGI app. It appears that IIS 6 may not implement http keep-alives for CGI. I haven't tried to prove it with ye-olde WireShark due to laziness, but if that's true, it is unfortunate, because the way Hg returns data over http for a pull requires keep-alives. I don't about IIS 7, since I don't have access to it anymore.

    I'm considering using the beta FastCGI add-on for IIS 5.1/6 or isapi_wsgi to see if I can make it work a little better. I'll write a new post once my experiment is complete.

    ReplyDelete
  3. Greetings.
    Has anyone had success pushing to a repository hosted on IIS 7 (Windows Server 2008)?

    I keep getting 'abort: HTTP Error 500: Internal Server Error'.

    Already configured:
    push_ssl = false
    allow_push = *

    ReplyDelete
  4. You should be able to view your webappdir.cgi in a web browser. They built in a web app to view your repos. If that doesn't work, my guess is your IIS install isn't configured correctly.

    You'll need to make sure you enable CGI for the python interpreter, which is surprisingly fussy, even on IIS 7. Go through this and double-check your settings.

    ReplyDelete
  5. *webappdir.cgi
    Err, I meant hgwebdir.cgi. I'm confusing myself. ;)

    ReplyDelete
  6. Greetings. I had no problems browsing or even cloning my repositories on the network and triple-checked my CGI configuration.

    My problem is when I try to push changes to my IIS-hosted repository. It gives me an 'abort: HTTP Error 500: Internal Server Error'.

    Have you been able to actually push via HTTP?

    Thanks again.

    ReplyDelete
  7. I followed your tutorial to setup HG on IIS6 and so far, so good. The problem is that I can't list repositories, although the CGI is working (it just shows up empty).

    the path to my repository is

    /test = d:/HG_repo/**

    now, if I create the same directory in c:, it works like a charm.

    For some reason it's ignoring the drive letter and I can't figure out why, any idea where should I start debugging this issue?

    ReplyDelete
  8. Try dropping the leading forward slash off the left hand side.

    Also, it is expecting you to have the right hand side pointed at a directory that contains repo directories, not directly at the repo itself.

    So you would do this:
    foo = D:\Repos

    where D:\Repos contains your HG_Repo directory, like this D:\Repos\HG_Repo .

    I'm guessing its like that so it can be more flexible for adding/removing repos, since the repo is considered pretty cheap in HG.

    ReplyDelete
  9. Tried that, forward, backward slashes or no slashes, it makes no difference. As soon as I put D: in front of that path, everything stops working.

    By the way, I am pointing to a container of repositories and I tested it by downloading the HG and the Mercurial book there. On c: everything works fine. I bet there is a problem with parsing of the path in either python or in mercurial.

    ReplyDelete
  10. Wouldn't surprise me in the least. Some of these Python-based projects aren't fully cross-platform in certain corner cases. :*(

    Why not try using an NTFS junction point and map your repos directory from your D onto your C drive? Your stuff will still physically live on D, but it would appear to reside on C at the same time.

    I know its hacky, but it should theoretically work.

    ReplyDelete
  11. I think I had an illumination: how do I figure out what's the user for the process that reads directories? Could it be that the user has permissions only for C: but not for D:?

    ReplyDelete
  12. Yep, tried using junction, it still doesn't work. I think it's a permission problem.

    Any idea how I fix that?

    ReplyDelete
  13. Ok, I'm big fat idiot...

    The user group that contains (possibly, I'm not IT admin) the IIS user didn't have read/write access to the repository folder on drive d:

    sometimes giant things are the hardest to see.

    ReplyDelete
  14. Greetings. I successfully managed to host repos on Windows XP (32 bit).

    Is there a way to allow authenticated push via http, instead of uisng the "allow_push = *" option?

    ReplyDelete
  15. I mean... first of all, authentication information is given by the web server? Or can it be stored in some configuration file, etc.?

    ReplyDelete
  16. IIS' built-in authentication depends on Windows' authentication services, so basically you have to create a Windows user account and push authenticating those credentials.

    In the hgrc file, I put

    allow_push = winusername

    and it worked. I could see certain circumstances where you may have to put DOMAIN\winusername or SERVER\winusername.

    ReplyDelete
  17. I double-checked that I installed everything according to the instructions given above, but I get the following error when I navigate to the cgi using a browser:

    The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are "Traceback (most recent call last): File "D:\HgWeb\hgwebdir.cgi", line 10, in <module> from mercurial import demandimport; demandimport.enable() ImportError: No module named mercurial ".

    Can anyone help?

    ReplyDelete
  18. Ok, I double-dog checked this time, and the problem was that I had extracted library.zip and copied templates from the TortoiseHg directory instead of the Mercurial directory.

    I was wondering why I had installed Mercurial :-)

    All is well.

    Thanks for these great instructions!

    -Corey

    ReplyDelete
  19. Ok, I thought I had everything working, but there is something really wrong going on.

    I have multiple repositories. Whenever I push to one of them, it pushes to ALL OF THEM even though I properly specify the URL: https://hg.myserver.com/hgwebdir.cgi/RepositoryName

    What is going on? This seems like a major bug!

    -Corey

    ReplyDelete
  20. Well, it appears to not fully push. It updates the meta data or whatever, so when I look at the repository explorer the information is there, but then the files are NOT THERE except only in the repository that was intended to be pushed to.

    Has anyone had real success doing more than one repository in the same web?:

    ReplyDelete
  21. SOLUTION (for those who run into this problem):

    You *CANNOT* specify the path to the repository with a drive letter on it.

    BAD:

    MyRepository = C:\HgRepositories\MyRepository
    AnotherRepository = C:\HgRepositories\AnotherRepository

    GOOD:

    MyRepository = \HgRepositories\MyRepository
    AnotherRepository = \HgRepositories\AnotherRepository

    This means that you HAVE to have the repositories on the same drive as the web. I did not want to do this, but I have to now. There is no way around this.

    The behavior was really weird. Using drive letters REALLY confuses it big-time. I could remove a repository from the above list so that the web did not know about it, I would then push a change to a known repository and the web would UPDATE BOTH ANYWAY!

    Anyway, be sure you are specifying the paths to the repositories correctly.

    I'm back in business.

    Enjoy!

    ReplyDelete
  22. In the browser i can see my repos and information about these. But if I try to clone, I get the HTTP Error 502: Bad Gateway. I don't know where the problem is. :(

    ReplyDelete
  23. I've triple checked all the settings above and it still doesn't work.

    when I use IE at 'http://mainpage.com/hgpage/hgwebdir.cgi' or any variant there of, I get either a 'PAge cannot be found' or '404' error message. I never see any repositories or any such thing.

    repo dir is 'd:\hg.mainpage\'
    virtual directory is 'd:\hg.project\'

    Win2003, IIS 6

    ReplyDelete
  24. Dan did you figure it out? I'm having the 502.2 - Bad Gateway (Windows 2008)

    ReplyDelete
  25. I got the same error "Bad Gateway" and solved it. Think it's about:
    1.Install Mercurial 1.2.1 on the server. The version that comes with TortoiseHg is not sufficient, it will not drive the CGI application.

    ReplyDelete
  26. I still don;t have it yet, but I did find that Python wasn't running at all, so I got that fixed.

    btw virtual directory names can't have decimals in them (hg.project) they have to be hgproject or such. That fixed the python problem.

    ReplyDelete
  27. My current error is:

    CGI Error
    The specified CGI application misbehaved by not returning a complete set of HTTP headers.

    ReplyDelete
  28. I was able to get this up and running. Just wondering if anyone has had any luck setting this up so that it will use the user's windows login credentials as the User ID when doing a commit? I am guessing I can turn on windows integrated security, but how would I configure so anyone can view, but only proper domain users can push?

    ReplyDelete
  29. I get a "Bad Gateway" exception when trying to run hgwebdir.cgi from a web application which exists on a different drive to my main system drive.

    ReplyDelete
  30. If you're using Mercurial 1.5 make sure you are using Python version 2.6.4 to run the cgi scripts in IIS.

    ReplyDelete
  31. Okay, so this is really getting on my nerves. I set up an HG repo online using this little guide, but every time I try to push to the online repo, it says "abort: HTTP Error 500: access is denied", yet if I go to the server where the repo is stored and hg update, the changesets I push have apparently gone through.

    Anyone got any ideas here? I'm pretty new to IIS.

    ReplyDelete
  32. I guess Python 2.5.4 works for Mercurial 1.2.1, but in the general case, you can find out what version of Python your current version of Mercurial was compiled against by right clicking python*.dll in your Mercurial install directory and choosing properties, then looking at the details tab. In my case, I've just upgraded to Mercurial 1.7.0, and had to install Python 2.6.4 to avoid a magic number error.

    ReplyDelete