Step 1: If you are on the machine that is running rlm.exe, run rlm.exe in admin mode
Step 2: Go to the browser and type http://localhost:5054. The RLM web server will look something like this:
Step 3: Then you can set it up as service by clicking in the "Manage Windows Service" on the left panel.
Step 4: Modify the following settings if required and click on Apply.
Step 5: Upon successful installation of the service you can restart the system to get that into effect.
Refer to the RLM Documentation (RLM Documentation - Copyright (C) 2006, Reprise Software, Inc) Page 13 on "Running the rlm server as a service on Windows" for more details.