Authentication on OpenSSH Windows 10 Host from OpenVMS Using Public Keys

Prerequisites

Environment

• OpenVMS V8.4 and above
• TCP/IP V5.7 (SSH.COM format)
• Windows 10 SSH server (OpenSSH format)

System Configuration

• Public key authentication is allowed on both hosts being connected.
• SSH Client is enabled on OpenVMS

  The client is enabled by calling the SYS$MANAGER:TCPIP$CONFIG.COM script and selecting 2. Client components → 7. SSH client from the menu.

• User with administrator privileges on Windows 10
• Running OpenSSH server on the Windows 10 host

  OpenSSH server is enabled by checking the appropriate checkbox from Settings → Apps → Apps & Features → Optional features.

  To start the server, one needs to open Computer Management → Services and Applications → Services, select OpenSSH SSH Server Service, and then click on Start the service. It is also recommended to change the startup type to Automatic to start the service upon Windows loading.

• OpenVMS Telnet connectivity to the Windows host running OpenSSH.

  To test connectivity, execute:

  $ TELNET OpenSSH 22

  A banner should be displayed in the terminal upon successful connection. If unable to connect, contact appropriate support.

Common Steps

1. Generate the key pair:

   $ set def sys$login                       ! This is for the account on VMS that will use public key authentication.

   $ set def [.SSH2]

   $ @sys$manager:tcpip$define_commands      ! The account must have appropriate privileges.

   $ ssh_keygen OpenVMS                      ! This command produces the keypair. Note the lack of extension on the target filename - OpenVMS.

        OpenVMS.                             ! Is the Private key

        OpenVMS.PUB                          ! Is the Public key

2. Verify the public key (example):

   $ ssh_keygen -"F" OpenVMS.pub

   Fingerprint for key:

   xemer-dovak-cevol-kukoh-fifav-zopub-sucil-tydic-gebar-nalen-cyxax

   If the fingerprint is not returned, then the public key is not usable. Try generating the keys again. Contact appropriate support for further assistance if it is still not working.

3. Populate the IDENTIFICATION. file :

   $ create IDENTIFICATION.                  ! Note the lack of a file extension

   IdKey OpenVMS

   [CTRL-Z] Exit

   dir IDENTIFICATION.;

   Directory SYS$SYSDEVICE:[user.ssh2]

   IDENTIFICATION.;1

   Total of 1 file.

4. Set protection on the files:

   $ SET FILE/PROT=(S,W,G,O:RE) IDENTIFICATION.

   $ SET FILE/PROT=(S,W,G,O:RE) OpenVMS.*

5. Transfer the .PUB file to the OpenSSH Windows 10 SSH server.

Method A

SFTP the .PUB to OpenSSH:

sftp user@OpenSSH

  user@OpenSSH's password:                       ! Note the password prompt

sftp> cd .ssh

/C:/Users/user/.ssh/

sftp>

sftp> put OpenVMS.PUB

OpenVMS.PUB | 1.2kB | 1.2 kB/s | TOC: 00:00:01 | 100%

sftp> quit

Method B

1. Copy/paste the contents of the public key from OpenVMS to OpenSSH on Windows 10:

   $ TYPE OpenVMS.PUB

   The user needs to be logged into an Administrator account on a Windows 10 PC. Files cannot be edited from a command line.

2. Open a Notepad session and paste the contents of the TYPE command output from above into a text editor, such as Notepad. Include the beginning and ending lines as well as all of the text between them:

   ---- BEGIN SSH2 PUBLIC KEY ----

       . text .

       . text .

       . text .

   ---- END SSH2 PUBLIC KEY ----

3. Save the Notepad session in the .ssh folder for the user as OpenVMS.PUB.
4. Open a CMD prompt window on the Windows 10 system to perform the following steps:
   1. Convert the SSH.COM format public key to an OpenSSH test file and verify the converted key:

      C:\Users\%USERNAME%\.ssh>ssh-keygen -i -f openvms.pub > vms2openssh.pub

      C:\Users\%USERNAME%\.ssh>ssh-keygen -B -f vms2openssh.pub

      The output of the second command should generate a fingerprint that contains the same output as the one that was generated on the OpenVMS system by $ ssh_keygen -"F" OpenVMS.pub from Step 2. If the fingerprint is not returned, the key is not usable. In this case, try Method A and/or Method B above again. If it is still failing, contact appropriate support for further assistance.

   2. Once verified, the vms2openssh.pub file can be deleted.
5. Convert the SSH.COM format key to the OpenSSH format and add it to the authorized_keys file. Older versions of OpenSSH used a file called authorized_keys2. Windows 10 uses a newer version of OpenSSH which changed the file name to authorized_keys:

   C:\Users\%USERNAME%\.ssh>ssh-keygen -i -f openvms.pub >> authorized_keys

6. Change permissions on the authorized_keys file in the users .ssh folder. Both SYSTEM and the user should have full control over the file.
   1. Open a File Explorer window and navigate to the .ssh folder.
   2. Right click on authorized_keys and go to Properties → Security → Advanced.
   3. Click Disable inheritance.
   4. Choose Convert inherited permissions into explicit permissions on this object when prompted. Remove all permissions from the file except for the SYSTEM and the user. There must remain exactly two permission entries on the file.
7. To enable the public key type of ssh-dss (needed to support the SSH.COM format key from the OpenVMS system) and to prevent public key access from trying to read the administrators_authorized_keys file in C:\PROGRAMDATA\ssh, the sshd_config file needs to be modified as follows:
   1. Right click and copy the sshd_config file from C:\PROGRAMDATA\ssh and save it to a folder local to the user (Documents, Downloads, etc.).
   2. Right click on the copied file and select Open with. Select Notepad or your preferred text editor/processor from the popup window and click OK.
   3. In the editor window, scroll down to the line that reads #PubkeyAuthentication yes. Enter the following line below it:

      PubkeyAcceptedKeyTypes=+ssh-dss

   4. Scroll to the bottom of the file and comment out the lines that read:

      Match Group administrators

          AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys

      by adding a # character at the beginning of each line. They should look like this:

      # Match Group administrators

      #    AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys

   5. Save the modified sshd_config file to the local user folder. Right click the saved file, select Copy, and then right click in the C:\PROGRAMDATA\ssh folder and select Paste. Click on Replace the file in the destination in the popup window. Click Continue in the Destination Folder Access Denied popup window that appears. This provides administrator permission to allow copying into the folder.
   6. Start or restart the OpenSSH SSH Server service on the Windows 10 system to have the service read the new sshd_config file.
8. Test SFTP from OpenVMS:

   $ sftp OpenSSH

   <<<NOTE THE LACK OF PASSWORD PROMPT>>>

   sftp> pwd

   /C:/Users/user

   sftp>