BhyveProject

Download



TroubleShoot / Frequently Asked Questions (FAQ)


I lost my account, HELP!

To invoke password recovery, run:
`/var/lib/nPulse/BVCP/Backend/vmm reset_password`

noVNC window is blinking and always reconnecting

Due in fact Bhyve allows one VNC session, please ensure that only one VNC window opened on the same Virtual Machine, otherwise the VNC transmission falpping between your browser windows.

How can I add .ISO images?

When the default installation path is /vms then you should see there a folder /vms/iso_images. Simply copy or move there.

Sometimes I experience network lag especially when I start a virtual machine

This could happen only when the very first VM attached into the bridged network, in this case the FreeBSD reset the network interfaces that involved.

Can I have bridged network interface?

Yes, absolutely out-of-the-box.

Windows is frozen, hung up

Windows VMs are could freeze because of legacy e1000 network driver, use virtio instead!

Linux, Debian does not boot

Bhyve UEFI firmware does not save the path after reboot, its a little tricky, but UEFI loader should be installed as `boot` prefix. To solve this issue we recommend to boot-up with livecd and rename debianx64 efi file to bootx64, and same with the debian folder should be boot. Learn More.

Can I have multiple hosts with one webinterface?

Yes, absolutely out-of-the-box, see below.

How can I start/stop/restart the frontend?

service bvcp-frontend start / stop / restart

How can I start/stop/restart the backend?

service bvcp-backend start / stop / restart

How can I start/stop/restart the helper?

service bvcp-helper start / stop / restart

Can I have NAT interface with portforwarding enabled?

Yes, absolutely but this is requires manual configuration, Learn more.

Where is the software installed?

The software is running under /var/lib/nPulse/BVCP untouched and by default: /vms. Because the software `integrates` with the OS but running independently.

Where will be the disk images created?

The software lists all of your mountpoints as drive, you can add them as `Storage` so then a new folder `vm_images` created into the root of the mountpoint here you can find your images after creation.

How to migrate, what disks are supported?

Migrating from existing virtual machines is very possible, please keep in mind BVCP only supports raw disk images so if you have different, like .vmdk, .qcow2 you must convert it into raw first. BVCP does not support other than UEFI!

Why FreeBSD?

Because somehow everything works much faster on FreeBSD regardless what we are trying to mess with Linux boxes.

Should I learn FreeBSD?

You should, Installing FreeBSD 10x harder than managing it, its just like as any linux distro!

Why this is better than others?

No one said that, but this is pretty different and I have not seen any working UI for Bhyve yet.

Does it IPv6 Ready?

It is. IPv4 and IPv6 are fully Supported!

What packages needed, depedencies?

Nothing just the FreeBSD!

Can I use it with Let's Encrypt?

Yes Of Corse, the self-signed SSL certificates are only generated if there is none! Please refer for /var/lib/nPulse/BVCP/sslCertificate.pem and bvcp.conf for more informations!

Can I run Windows with it?

Yes, but UEFI capable windows (Windows 7, Windows 10) and only with virtio drivers.
Please note that the latest release of BVCP supports NVME storages which is natively works on Windows 10+.
Windows 11 requires TPM support that does not found on Bhyve and should not be, but many workarounds already exists on the internet to solve this easily, even Microsoft let this workaround happen.

Linux wont boot, help!

Due in fact BVCP uses UEFI bootcode, some linux distributions like Debian 10 install UEFI boot image under debian prefix, this is easily can fixed: Learn more.

Does it sending usage statistics?

Yes. To be fair we only get error reporting and initial info such as hostname, OS version, License status.

Summary

BVCP is a robust all-in-one set for managing Bhyve Virtual Machines on FreeBSD via secure webinterface.
BVCP uses a lightweight webinterface, supports:

  • - Authentication with detailed logging
  • - System Health Assessments
  • - TLS/SSL
  • - noVNC Console
  • - User Management
  • - Storage Management
  • - Network Management
  • - VM Management

Frontend WebUI

Frontend running with unprivileged (www) user and does not interact with the system directly, every call uses built-in API through AES128 point-to-point encryption.

Frontend uses its own built-in webserver, supported protocols are: HTTP/1.0 HTTP/1.1 (TLS/SSL), no third party module needed.

Frontend/Authentication

The login interface provided by the frontend, but the authentication itself is happening on backend side always, the software uses cookies to store login information as encrypted form and revalidates every x seconds.

Security Fencing

This model proven as very secure within the past years, hence an attacker can not interact with a system-wide process directly nor the database, in fact the frontend also does not have database connection.
Users can interact with the Virtual Machines only, and no way to touch the main OS from this software.

DOS/DDOS

As inherited from a security appliance, our framework automatically managing the incoming connections, so in case of abnormal traffic the DOS mitigation kicks in and ignoring, that is very effective however not a full protection against DOS/DDOS.

noVNC Console

Users can interact graphically with the virtual machines, due the VNC protocol is unsafe due its unencrypted, we encapsulates into TLS traffic between the Backend - Frontend - EndUser line.

When using VNC Connection, users will connect to the frontend, then the frontend make a backside connection into the backend.

The Backend

The backend is running with highest (root) privileges and serve a TCP servers for RFB (VNC) and for the API.
Every operation done by backend.

Utility

The backend has some power-utility, such as reset admin password, and setup / update API CLI.

Backend / Helper

Backend running twice, the first instance will be the backend server for API/RFB interface, and the second one is the helper. The helper start VM for example and this is important due in some circumstances the VM will inherit open filedescriptors from the caller task.

Database

BVCP uses SQLite!

VMCTL

VMCTL is a very small single C program that makes enable us to interact with bhyve and store return codes, if we doing this from the backend then all virtual machine could be killed if the backend process killed.

Alternatives, Similar Softwares

- ProxMox/VE
- VMWare Esxi
- Oracle VirtualBox

Configuration File

/var/lib/nPulse/BVCP/bvcp.conf

; Configuration File

api
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 1 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 127.0.0.1 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = ::1 ; IPv6 Listening Address, use: :: to bind on all interfaces
        max_connections_per_ip = 50 ; Defines maximum allowed connections per IP Address
        max_data_size = 200 ; Maximum allowed data is 200MB
        port = 8628 ; Port number that listening on
}

core
{
        date_format = %Y/%m/%d ; Defines Date format C-Function 
        time_format = %H:%M:%S ; Defines Time format C-Function 
        version = v12 ; Config Version
}

geoip
{
        asn_database = geoIP/geolite_asn.mmdb
        city_database = geoIP/geolite_city.mmdb
        country_database = geoIP/geolite_country.mmdb
}

rfb
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 1 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 127.0.0.1 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = ::1 ; IPv6 Listening Address, use: :: to bind on all interfaces
        max_connections_per_ip = 50 ; Defines maximum allowed connections per IP Address
        max_data_size = 200 ; Maximum allowed data is 200MB
        port = 8659 ; Port number that listening on
}

security
{
        api_key = 208C694F9CBD2BFA47F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB93BDC ; SECRET! API KEY
        secret = 59DFDCCC370406476578BDB1A42F2E0A05113D161F3F6A931E78564A7D034EC2  ; SECRET, INTERNAL USE ONLY
}

vm
{
        vm.hostname = master.bhyve.npulse.net ; Self Hostname, important to matches with real hostname
        vm.root = /vms ; Data Dir, where the log files and database is located.
}

vmctl
{
        auth_node = master.bhyve.npulse.net ; Defines the master node, on multiple nodes this node refer for users and authentications
        dns_ip = 8.8.8.8 ; DNS IP for built-in DNS resolver
        nodes = master.bhyve.npulse.net one.bhyve.npulse.net ; Phyisical Nodes
        smtp_auth_user =  ; SMTP Auth User for mailing support
        smtp_from =  ; SMTP Sender Address for mailing support
        smtp_password =  ; SMTP Password for mailing support
        smtp_port = 25  ; SMTP Port for mailing support
        smtp_server =  ; SMTP Server for mailing support
        two_factor = yes  ; NOT IMPLEMENTED
}

vmctl_master.bhyve.npulse.net ; Delegated Configuration for node
{
        host = localhost ; Physical Host Name or IP Address
        key = 208C694F9CBD2BFA47F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB93BDC ; API KEY
        name = Master Node ; Name that will shown
        port = 8628 ; API Port
        rfb_port = 8659 ; RFB/VNC Port
}

vmctl_one.bhyve.npulse.net ; Second node, Always prepend 'vmctl_'
{
        host = 91.82.18.18 ; Physical Host Name or IP Address
        key = AAAA694F9CBD000047F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB9F0DC ; API KEY
        name = The One ; Name that will shown
        port = 8628 ; API Port
        rfb_port = 8659 ; RFB/VNC Port
}

webui
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        enable_compression = true ; Enable GZIP / DEFLATE over HTTP
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 4 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 0.0.0.0 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = :: ; IPv6 Listening Address, use: :: to bind on all interfaces
        keep_alive_ms = 60000 ; HTTP Keep-Alive timeout after 60 seconds
        max_connections_per_ip = 200 ; Defines maximum allowed connections per IP Address
        port = 8086 ; Port number that listening on
        post_max_size_mb = 200 ; Maximum POST data is limited to 200MB
        setuid_user = www ; Set-UID user
        ssl_cert = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate Location
        ssl_chain = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate CA-Chain Location
        ssl_key = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate Key Location
        static_cache_sec = 3600 ; Static files cached for one hour
        use_ssl = true ; Enable and Forces SSL
}