BhyveProject

Download



TroubleShoot / Frequently Asked Questions (FAQ)


I lost my account, HELP!

To invoke password recovery, run:
`/var/lib/nPulse/BVCP/Backend/vmm reset_password`


noVNC window is blinking and always reconnecting

Due in fact Bhyve allows one VNC session, please ensure that only one VNC window opened on the same Virtual Machine, otherwise the VNC transmission falpping between your browser windows. Prior BVCP 2.0.0 this should not be issue anymore.


It says the VM is rebooting but nothing happens

In this case the VM could not be started properly, maybe not enough of memory available, check your dashboard and also you can check the bhyve output at default `/vms/tmp/[VN_NAME]/log` location.


How can I add .ISO images?

When the default installation path is /vms then you should see there a folder /vms/iso_images. Simply copy or move there.


Sometimes I experience network lag especially when I start a virtual machine

This could happen only when the very first VM attached into the bridged network, in this case the FreeBSD reset the network interfaces that involved.


Can I have bridged network interface?

Yes, absolutely out-of-the-box.


Windows is frozen, hung up

Windows VMs are could freeze because of legacy e1000 network driver, use virtio instead!


Linux, Debian does not boot

Bhyve UEFI firmware does not save the path after reboot, its a little tricky, but UEFI loader should be installed as `boot` prefix. To solve this issue we recommend to boot-up with livecd and rename debianx64 efi file to bootx64, and same with the debian folder should be boot. Learn More. Prior BVCP 2.0.0 this should not be issue anymore.


Can I have multiple hosts with one webinterface?

Yes, absolutely out-of-the-box, see below.


How can I start/stop/restart the frontend?

`service bvcp-frontend start / stop / restart`


How can I start/stop/restart the backend?

`service bvcp-backend start / stop / restart`


How can I start/stop/restart the helper?

`service bvcp-helper start / stop / restart`


Can I have NAT interface with portforwarding enabled?

Yes, absolutely but this is requires manual configuration, Learn more.


Where is the software installed?

The software is running under /var/lib/nPulse/BVCP untouched and by default: /vms. Because the software `integrates` with the OS but running independently.


Where will be the disk images created?

The software lists all of your mountpoints as drive, you can add them as `Storage` so then a new folder `vm_images` created into the root of the mountpoint here you can find your images after creation.


How to migrate, what disks are supported?

Migrating from existing virtual machines is very possible, please keep in mind BVCP only supports raw disk images so if you have different, like .vmdk, .qcow2 you must convert it into raw first. BVCP does not support other than UEFI!


Why FreeBSD?

FreeBSD has smaller resource footprint and faster. Thats means you can run more virtual machines with more modern technology.


Should I learn FreeBSD?

You should, Installing FreeBSD 10x harder than managing it, after that its just like as any linux distro! Commands are nearly the same. And if you watch one of the most popular streaming service or playing with PlayStation, you are already a FreeBSD user by now.


Why is this better than others?

No one said that, but this is pretty different and I have not seen any working UI for Bhyve yet. After years I can say that, this one is reliable and just works, that could be a reason.


Does it IPv6 Ready?

It is. IPv4 and IPv6 are fully Supported!

What packages needed, depedencies?

Nothing just the FreeBSD!

Can I use it with Let's Encrypt?

Yes Of Corse, the self-signed SSL certificates are only generated if there is none! Please refer for /var/lib/nPulse/BVCP/sslCertificate.pem and bvcp.conf for more informations!

Can I run Windows with it?

Yes, but UEFI capable windows (Windows 7, Windows 10) and only with virtio drivers.
Please note that the latest release of BVCP supports NVME storages which is natively works on Windows 10+.
Windows 11 requires TPM support that does not found on Bhyve and should not be, but many workarounds already exists on the internet to solve this easily, even Microsoft let this workaround happen.

Does it sending usage statistics?

Yes. To be fair we only get error reporting and initial info such as hostname, OS version, License status. We are regulated and we are comply with both EU and US laws. For more information please visit https://npulse.net/en/about/19-our-privacy--data-handling-policy


Summary

BVCP is a robust all-in-one set for managing Bhyve Virtual Machines on FreeBSD via secure webinterface.
BVCP uses a lightweight webinterface, supports:

  • - Authentication with detailed logging
  • - System Health Assessments
  • - TLS/SSL
  • - noVNC Console
  • - User Management
  • - Storage Management
  • - Network Management
  • - VM Management

Frontend WebUI

Frontend running with unprivileged (www) user and does not interact with the system directly, every call uses built-in API through AES128 point-to-point encryption.

Frontend uses its own built-in webserver, supported protocols are: HTTP/1.0 HTTP/1.1 (TLS/SSL), no third party module needed.

Frontend/Authentication

The login interface provided by the frontend, but the authentication itself is happening on backend side always, the software uses cookies to store login information as encrypted form and revalidates every x seconds.

Security Fencing

This model proven as very secure within the past years, hence an attacker can not interact with a system-wide process directly nor the database, in fact the frontend also does not have database connection.
Users can interact with the Virtual Machines only, and no way to touch the main OS from this software.

DOS/DDOS

As inherited from a security appliance, our framework automatically managing the incoming connections, so in case of abnormal traffic the DOS mitigation kicks in and ignoring, that is very effective however not a full protection against DOS/DDOS.

noVNC Console

Users can interact graphically with the virtual machines but the VNC protocol is unsafe due its unencrypted, we encapsulates into TLS, hardened AES as traffic between the Backend - Frontend - EndUser line.

When using VNC, users will connect to the frontend, then the frontend make (secure, AES) tunnel into the backend. So the VNC data is encrypted between any party.

The Backend

The backend is running with highest (root) privileges and serve a TCP servers for RFB (VNC) and for the API.
Every operation done by backend.

The Helper

The helper is running with highest (root) privileges and start / stop the VMs. This is for security, because forked applications sometimes has access to it's parent. We focusing on that security, meanwhile others does not even care to run the frontend with unprivileged user, thats why BVCP born.


Utility

The backend has some power-utility, such as reset admin password, and setup / update API CLI.

Database

BVCP uses SQLite Database!

VMCTL

VMCTL is a very small single C program that just fork bhyve instance and keep it running.

Alternatives, Similar Softwares

- ProxMox/VE
- VMWare Esxi
- Oracle VirtualBox

Configuration File

/var/lib/nPulse/BVCP/bvcp.conf

; Configuration File

api
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 1 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 127.0.0.1 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = ::1 ; IPv6 Listening Address, use: :: to bind on all interfaces
        max_connections_per_ip = 50 ; Defines maximum allowed connections per IP Address
        max_data_size = 200 ; Maximum allowed data is 200MB
        port = 8628 ; Port number that listening on
}

core
{
        date_format = %Y/%m/%d ; Defines Date format C-Function 
        time_format = %H:%M:%S ; Defines Time format C-Function 
        version = v12 ; Config Version
}

geoip
{
        asn_database = geoIP/geolite_asn.mmdb
        city_database = geoIP/geolite_city.mmdb
        country_database = geoIP/geolite_country.mmdb
}

rfb
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 1 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 127.0.0.1 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = ::1 ; IPv6 Listening Address, use: :: to bind on all interfaces
        max_connections_per_ip = 50 ; Defines maximum allowed connections per IP Address
        max_data_size = 200 ; Maximum allowed data is 200MB
        port = 8659 ; Port number that listening on
}

security
{
        api_key = 208C694F9CBD2BFA47F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB93BDC ; SECRET! API KEY
        secret = 59DFDCCC370406476578BDB1A42F2E0A05113D161F3F6A931E78564A7D034EC2  ; SECRET, INTERNAL USE ONLY
}

vm
{
        vm.hostname = master.bhyve.npulse.net ; Self Hostname, important to matches with real hostname
        vm.root = /vms ; Data Dir, where the log files and database is located.
}

vmctl
{
        auth_node = master.bhyve.npulse.net ; Defines the master node, on multiple nodes this node refer for users and authentications
        dns_ip = 8.8.8.8 ; DNS IP for built-in DNS resolver
        nodes = master.bhyve.npulse.net one.bhyve.npulse.net ; Phyisical Nodes
        smtp_auth_user =  ; SMTP Auth User for mailing support
        smtp_from =  ; SMTP Sender Address for mailing support
        smtp_password =  ; SMTP Password for mailing support
        smtp_port = 25  ; SMTP Port for mailing support
        smtp_server =  ; SMTP Server for mailing support
        two_factor = yes  ; NOT IMPLEMENTED
}

vmctl_master.bhyve.npulse.net ; Delegated Configuration for node
{
        host = localhost ; Physical Host Name or IP Address
        key = 208C694F9CBD2BFA47F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB93BDC ; API KEY
        name = Master Node ; Name that will shown
        port = 8628 ; API Port
        rfb_port = 8659 ; RFB/VNC Port
}

vmctl_one.bhyve.npulse.net ; Second node, Always prepend 'vmctl_'
{
        host = 91.82.18.18 ; Physical Host Name or IP Address
        key = AAAA694F9CBD000047F8E4EC7C0D2A5FB3B29984802E3E049A73A2011CB9F0DC ; API KEY
        name = The One ; Name that will shown
        port = 8628 ; API Port
        rfb_port = 8659 ; RFB/VNC Port
}

webui
{
        auto_blacklist_sec = 10 ; 10 seconds of blacklist if one IP exceeds max_connections_per_ip
        enable_compression = true ; Enable GZIP / DEFLATE over HTTP
        idle_timeout_ms = 60000 ; Connection will be dropped if no transmission done within 60 seconds
        instances = 4 ; Defines how many instances run at once, consume more CPU and RAM
        ipv4_listen = 0.0.0.0 ; IPv4 Listening Address, use: 0.0.0.0 to bind on all interfaces
        ipv6_listen = :: ; IPv6 Listening Address, use: :: to bind on all interfaces
        keep_alive_ms = 60000 ; HTTP Keep-Alive timeout after 60 seconds
        max_connections_per_ip = 200 ; Defines maximum allowed connections per IP Address
        port = 8086 ; Port number that listening on
        post_max_size_mb = 200 ; Maximum POST data is limited to 200MB
        setuid_user = www ; Set-UID user
        ssl_cert = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate Location
        ssl_chain = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate CA-Chain Location
        ssl_key = /var/lib/nPulse/BVCP/sslCertificate.pem ; Defines TLS Certificate Key Location
        static_cache_sec = 3600 ; Static files cached for one hour
        use_ssl = true ; Enable and Forces SSL
}