Tag Archives: homepage

Setting up HiveOS on ESXi 6.7 guest using GPU passthrough

This guide is to give a quick overview on how to setup HiveOS on ESXi 6.7 using PCI passthrough to setup a GPU. This guide assumes that you have an ESXi 6.7 (or higher) setup working. In this case I have the limitation of using vmklinux driver to power up my AMD Ryzen setup, so I cannot upgrade to ESXi 7 without acquiring a new NIC.

Also, this test was done only with a ZOTAC RTX 3070 but all the cards should work exactly the same way. Not sure about PCI passthrough limitations on VMware but in this case it’s only one GPU, so YMMV. I’m using the free version.

This guide is specific to the deployment of HiveOS, however this can be leveraged for other workloads that you might want to virtualize such as Tensorflow. This won’t cover GPU performance tuning, underclocking, etc. as I don’t have the proper experience with HiveOS and mining. No performance should be lost using DirectPath I/O but I’m unsure if this actually true as I didn’t test this device on a non virtualized environment (yet!). However, seems like common consensus about this card is around the same output as the one I’m getting (61.90 MH/s).

Step 1:  Download HiveOS
Once you download it the format will be on an .img file and you’ll need to convert it to VMDK. I used Starwind V2V converter for this task.

After converting it I had an issue after I tried to attach it to a VM, so I had to clone the disk inside of ESXi to make it work. (I also tried to upload it from VMware Workstation without success).

For doing this, upload the .vmdk file to your datastore and enable SSH on your ESXi host.

Syntax:

vmkfstools -i /vmfs/volumes/datastore/filename.vmdk /vmfs/volumes/datastore/newfilename.vmdk

Example:

[root@homelab:~] vmkfstools -i /vmfs/volumes/datastore-hdd-1/hiveos1/[email protected] /vmfs/volumes
/datastore-hdd-1/hiveos/hiveos.vmdk

Once this is completed after few minutes depending on your SSD/HDD, the clone will finish and you should be able to assign this drive to a VM HD.

Step 2: Setup your PCI device as passthrough
Make sure that your passthrough status looks as active. If its inactive, click on the upper button “Toggle passthrough”.

This step requires a reboot of the ESXi host.

Step 3: Customize your VM

First, expand memory and check the Reserve all guest memory (all locked) option.

Assign the PCI devices to the VM. Not sure if the sound is required (best guess is not) but I added it anyway.

Click on Add other device -> PCI device a couple of times and then assign both devices that come from your NVIDIA GPU.

After this, go to VM options -> Advanced -> Edit configuration and make sure this settings are in place (or set them if not).

svga.guestBackedPrimaryAware = TRUE
svga.present = TRUE
hypervisor.cpuid.v0 = "FALSE"

Please note that if any of this settings is not in place (specially the last one), the VM won’t be able to utilize the CUDA cores as it will complain about “invalid device ordinal”.

Step 4: Setup HiveOS

You should be all set to boot  your VM and do a regular HiveOS install. You need to go to HiveOS website and create a worker, you’ll acquire a worker ID and password which you need to setup on your VM.

After that you need to create a “Flight Sheet” if you don’t have one already. Personally I used PhoenixMiner as Ethminer pool wasn’t working at the time I wrote this guide.

This should be how it looks like after:

Now you can overclock as you like, without any modification it was doing 51 MH/s (having a 214w consumption) in average, after a little bit of tweaking and trying to be more efficient, I was able to get an average of 61.90 MH/s (125w consumption). But with this YMMV.

As always, feel free to reach out in case you have any question regarding this setup and I’ll try to answer.

Learning SAP – Part #1 – Installing ABAP on RHEL8

Lately I been sorta involved on doing things related to SAP so I thought it would be a great idea to try to setup ABAP at the home lab and test it out.

Full disclosure, I don’t have an actual clue of what’s going on and what I’ll do with this but I thought it was a good idea to try to learn. I’ll do follow up posts to go deeper into it (if I don’t get distracted with something else).

For this specific deployment my environment is the following:

  • VMware ESXi 6.7.0
  • Red Hat Enterprise Linux 8.3
  • SAP NetWeaver AS ABAP Developer Edition 7.52 SP04
  • VM sizing it’s 4 vCPU’s and 8GB RAM, 100GB partition for the whole OS.

Step 1: Get your OS deployed and updated.
This part it’s very straight-forward, go to developers.redhat.com and get yourself a free subscription for it. I did a minimal installation on it, subscribed and ran a yum update. (reboot!).

Once it’s deployed, you’ll require rpmfusion non-free (or download unrar from rarlab.com) as this is not providen by Red Hat or EPEL.

yum install libaio tcsh uuid libnsl

Also SAP has a requirement on UUIDD to be able to sort out your license properly, so it’s recommended to install it. However, this package is not available without a valid Red Hat for SAP subscription. You are able to get it from CentOS, I found it on rpmfind.

As this is just for testing, I won’t go too deep into how to setup the filesystems, but base requirements according to the documentation included in the binaries. If you require this on separate partitions, do it in advance.

File system recommendation
/sapmnt ~2GB
/sybase ~50GB
/usr/sap ~3G

My setup
/ 70G
/home 30G

Be sure of having your hostname on /etc/hosts as not having it will have you on a situation where no profile is found. (if this happen, just su to npladm and run startsap ALL manually after fixing it).

Step 2: Configure firewalld

You can skip this if you have firewalld disabled.

By default RHEL8 comes with firewalld using public as the default zone (see your zone executing firewalld --get-default-zone). Modify this while adding the service if you have a different setup.

# firewall-cmd --permanent --new-service=sapnpl
# firewall-cmd --permanent --service=sapnpl --add-port=3200/tcp
# firewall-cmd --reload
# firewall-cmd --permanent --zone=public --add-service=sapnpl
# firewall-cmd --reload

At this point, everything should be setup for you being able to open your SAP instance from SAP Logon on the St

Step 3: Go to developers.sap.com and create an account there for being able to download the trial versions of their products;

Step 4: Download ABAP, the version I’m downloading it’s 11 parts of about 13.5G total. You also need to download the license for it;

Step 5: Upload the SAP install to your VM. For this setup I will use /opt/sapmedia and use unrar to extract all of your files.

Be sure of having your license there as well.

ABAP files

Remove this files BEFORE running the installer! As it will continue to extract more files and you might run out of space.

Step 6: Run the SAP ABAP installer.

chmod +x install.sh
./install.sh

If all the dependencies were there, it should show you the SAP license agreement and ask you for the OS users that are going to be setup.

It will take a while here, so go grab a snack while that happens. By a while, I mean that you can actually go to the store to get the snack and begin next step once you’re back.

Step 7: Install the SAP Logon (I did install 760) on the client.

Step 8: Configure the system entry on SAP Logon and login.

Fill this up with the application server IP / DNS that your computer can resolve back to your server.

SAP profile
SAP profile setup

Once that’s done, login using the default passwords which you can find on /opt/sapmedia/index.html (if you extracted the files at /opt/sapmedia)

SAP Login
SAP Login as DEVELOPER

You can do the same as SAP* or DDIC . Default password for those two accounts is Down1oad.

Step 9: Setup SAP as systemd unit (optional)

By itself, the installer setup sapinit on init.d. However this doesn’t handle the start/stop of your SAP instance.

Some basic commands before we start setting up the systemd unit, so become doing # su - npladm (or login using the master password that was setup during the deployment).

At this point, if you restart the server the application won’t come up by itself, unless you start it manually.

Below the commands for checking the status of your ABAP instance, start and stop.

Using startsap/stopsap (deprecated)

$ startsap ALL
$ startsap check
$ stopsap ALL

Using sapcontrol

$ sapcontrol -nr 0 -function GetProcessList
$ sapcontrol -nr 0 -function StartSystem
$ sapcontrol -nr 0 -function StopSystem

Now we will create few files to make the systemd service, and yeah, there are way better ways to do this with systemd so I hope to hear back from you on the comments! Get as fancy as you can.

/usr/sap/NPL/nplstart.csh

#!/bin/csh
/usr/sap/NPL/SYS/exe/uc/linuxx86_64/sapcontrol -nr 0 -function StartSystem

/usr/sap/NPL/nplstop.csh

#!/bin/csh
/usr/sap/NPL/SYS/exe/uc/linuxx86_64/sapcontrol -nr 0 -function StartSystem

And finally, the service file at /etc/systemd/system/sapnpl.service

[Unit]
Description=sapnpl
After=sapinit.service multi-user.target
Requires=user.slice

[Service]
KillMode=none
Type=oneshot
User=npladm
Group=sapsys
ExecStart="/usr/sap/NPL/nplstart.csh"
ExecStop="/usr/sap/NPL/nplstop.csh"
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

and let’s make the first 2 files executables and give the ownership to npladm:sapsys

chown npladm:sapsys /usr/sap/NPL/nplstart.csh /usr/sap/NPL/nplstop.csh
chmod +x /usr/sap/NPL/nplstart.csh /usr/sap/NPL/nplstop.csh

Now you should be able to enable/start/stop the service. From this point and on, try not to mess around doing that not using systemd.

Below are the commands to deal with this:

$ systemctl enable --now sapnpl
$ systemctl status sapnpl
$ systemctl start sapnpl
$ systemctl stop sapnpl

The status will look like this

systemctl status sapnpl output
systemctl status sapnpl output

As I mentioned before, not really sure on what I’m trying to achieve or what I’ll do next with it and I’ll try to do a follow up on this soon.

Convert to RHEL from CentOS using Convert2RHEL

Hi everyone – It’s been a while since my last post and basically my situation is the following ben using CentOS 7 for a while at home and at some point on 2020 I did upgrade to CentOS 8 which probably I should have done a post about it is a more complex activity but well, I didn’t (yet).

Given the recent news of CentOS not following their standard cadence and moving away from what enterprise Linux is to CentOS Stream, you might want to move some workloads from CentOS to something with long-term support.

As you might know if you’re here, that includes Oracle Linux and the upcoming Rocky Linux (which first release is expected by end of March 2021) for some sort of easy conversion without re-deployment. Ubuntu is also trying to push new users towards their platform but that’s a bigger leap, specially for enterprise or to users that are not familiar with Debian derivatives.

Before doing any of this, please have the following considerations:

  • Convert2RHEL is currently unsupported by Red Hat;
  • Conversion on UEFI systems is not officially supported as there is an open bug on it, but I do address how to prevent (or how to fix) the issue below.
  • This should work on CentOS 6 as well, but this is something I did not test and as it is EOL, I won’t address it here.
  • Having a working backup of the server and that you have the means to restore it. Personally, I use ESXi so I’ll just take an snapshot;
  • Having a valid Red Hat subscriptions. You can obtain a free one thru the developer program here. They offer 16 VM’s which should be sufficient for most of home lab environments.
  • Perform a yum/dnf update to latest version before starting.
For CentOS 8:
# dnf install -y https://github.com/oamg/convert2rhel/releases/download/v0.17/convert2rhel-0.17-1.el8.noarch.rpm

For CentOS 7:
# dnf install -y https://github.com/oamg/convert2rhel/releases/download/v0.17/convert2rhel-0.17-1.el7.noarch.rpm

Initialize conversion:
# convert2rhel --username your_RHSM_account

At this point you’ll be asked to accept the EULA and to confirm a few times about moving forward with the conversion until you’re asked about your RHSM information and you need to select the subscription number which you want to utilize for this (it means the number on top of the subscription information, this is not the pool id!).

Once the conversion finishes, DO NOT REBOOT if you are on a UEFI system, as you’ll likely encounter (at the time of writing) the bug #1898314. (If you got to this article because you got an issue booting after it and landing on grub, don’t worry, I got you.)

If you didn’t reboot, great! You are in time to fix grub and be on your way to boot your system properly.

Step 1: Generate a grub2-mkconfig by doing

# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

 

Step 2: Delete the CentOS entry from efibootmgr

# efibootmgr -v

 

Now we need to delete the entry for CentOS Linux, in this case Boot004. You need to know the boot disk and partition (/boot/efi) for the following steps. (In my case it is on /dev/sda1). Change the release to 7 or 8 depending on what OS you use.

# efibootmgr -Bb 0003
# efibootmgr -c -d /dev/sda -p 1 -l \\EFI\redhat\shimx64.efi -L "Red Hat Enterprise Linux <version>"

At this point you should be having a working environment! Congratulations.

But if you did the same mistake I did… and here we are and rebooted right away after the conversion to find a grub> screen, this is because UEFI conversions are not supported. Basically you have two choices at this point:

  1. Fix it;
  2. Go back to snapshot/backup and re-do it.

If you go for the fix it way, you’ll need a CentOS or Red Hat disk/ISO and go to recovery mode. Once you are chrooted into /sysroot, you can go back to step 1 on how to fix it before rebooting and get it done. Once you’re done, exit the chroot environment and recovery mode and you should be able to login to your Red Hat installation if your upgrade was successful.

Personally I have done few of this upgrades and I haven’t encountered any major issue. Feel free to comment on your own experiences or ask any question below on the comment section.

Good luck!

Reference: redhat.com