With its help, your customers can manage virtual servers, backups, firewall as well as access VNC/SPICE console and much more - all this directly in your WHMCS client area.
Additionally, the module provides support for Cloud-Init for the KVM virtualization type and lets you conveniently manage all existing servers, IP addresses and clusters through the unique Proxmox Addon attached to the module.

We will guide you step by step through the whole installation and configuration process.

Note: You can check the current PHP version in your WHMCS. To do so, proceed to 'Utilities' → 'System' → 'PHP Info'.

The content of the PHP version files should look like this.

Note: If you wish to allow noVNC consoles, you need to configure a product using the PVE realm and set user permissions to 'PVEVMUser.'

The file is located in 'modules/servers/proxmoxVPS/license_RENAME.php'. Rename it from 'license_RENAME.php' to 'license.php'.

Enter your license key between the quotation marks as presented on the following screen.
You can find your license key in our client area → 'My Products'.

To do so, log in to your WHMCS and press 'Setup' → 'Addon Modules'. Find 'Proxmox Addon' and press 'Activate'.

Note: The Proxmox addon version might not match the current version of the provisioning module itself. This is completely normal due to the specific structure of the module.

To do so, click on the 'Configure' button, tick 'Full Administrator' and press 'Save Changes'.

Go to 'Setup' → 'Products/Services' → 'Servers' and press the 'Add New Server' button.

Choose an authentication type from 'Authentication' and 'Proxmox VPS' from the 'Type' dropdown menus. 'Save Changes'.

In order to check connection with a Proxmox server, simply press 'Test Connection' as shown on the screen below.

Click on 'Create a New Group'.

To create a product, click on 'Create a New Product'.

'Product Type' can be set to 'Dedicated/VPS Server' but nameservers will need to be added later.
If you do not want to fill out nameservers, please select 'Hosting Account' .

Next, 'Save Changes'.

Available virtualization types:

• KVM
  
• LXC (supported by Proxmox VE 4.0 and later)
  
• OpenVZ (supported by Proxmox VE 3.4 and earlier, prior to Proxmox v2.3.0)
  

You can also specify how often the bandwidth usage should be counted.
Afterward, press 'Save Changes'.
Note: If you choose the 'Auto-Node' as your 'Default Note', new virtual servers will be created on the least used node.

From dropdown menus choose 'KVM OS Type' and 'Default Storage'.
Type your VM description in 'Container Description' and scroll down.
You can also define VM prefix by filling out the 'Default KVM Container Prefix' field.
Set 'Use Client name For VPS' to 'Yes' if you want to use either the client's company name or their first/last name as the hostname value.
Here you may also decide whether you wish to use DHCP software to automatically assign IP addresses. If so, simply select the 'ISC DHCP Server' option. Read more about its installation here.

Additionally, set up backups in the 'Backups' section by choosing values from the 'Storage' and 'Backups Routing' dropdown menus.
Backups routing automatically removes the oldest backup as soon as a new backup is created and the backup limit has been reached.
Type in the desired number of days in the 'Store The Backup For N Days' field to remove backups older than the provided number.
In the 'Firewall Options' section you can set up available interfaces and the limit of firewall rules.
Important: Keep in mind that the root password is generated automatically for KVM virtualization upon its creation and reset.
If you want to set your own password on KVM virtualization, consult the Configuration of SSH Keys section.

From dropdown menus choose 'Default Storage' , 'Template' and 'Bridge'.
Enter your virtual servers description in 'Container Description' and press 'Save Changes'.
Remember! This virtualization type is supported only by Proxmox VE 4.0. and later
If you are using older version of a Proxmox server, you will not be able to start this virtualization.

Please note that OpenVZ virtualization is supported only by Proxmox versions prior to v.2.3.0

From the dropdown menus choose 'Default Storage' and 'OpenVZ OS Template'.
Additionally, define backup settings in the 'Backups' section by choosing values from the 'Storage' and 'Backups Routing' dropdown menus.
Backups routing automatically deletes the oldest backup when a new backup is created and a backup limit has been reached.
Fill out the desired number of days in the 'Store The Backup For N Days' field to remove backups older than the provided number.
Type your VM description into 'Container Description' and press 'Save Changes'.

Note that while generating 'Configurable Options' it is advisable to delete all options which you no longer use, e.g. for the KVM virtualization delete the swap option or IPv6.

php -q /your_whmcs/modules/servers/proxmoxVPS/cron/cron.php 

Do not forget to replace 'your_whmcs' with your WHMCS root location. The cron can also be run every single minute in order to perform the action more often.

In order to create a new product, you have to add 'Dedicated IP ' to the 'IP Addresses' list in 'Proxmox' addon module.
To do so, go to 'IP Management' and press 'Add Pool'.
Note: You can add both the IPv4 and IPv6 addresses.

Next, choose a server from the 'Server' dropdown menu and define which virtualization it should be applied to through selecting it from the 'Virtualization' dropdown menu.
Afterward, press 'Add'.
Note! Automatic IP assignment works only for OpenVZ/LXC machines.
KVM virtualization does not support IP Management provided by the module. Instead, IP can be assigned via the console.

If you are using the KVM virtualization type, the client area with a successfully created product should look similar to the one on the screen below.

SSH keys are required to enable the automatic resizing of partition upon the disk upgrade as well as the change of passwords for the KVM virtualization type.
Note - 'Resize Partition', and 'Change Root Password' will only work for newly created VMs with ISC DHCP and Internet access configuration.
We recommend using this option only for new products.

Then, generate SSH keys and add the public key to the root account on the newly created VM image. Moreover, upload both ( public and private ) keys on your WHMCS server in a secure location.

Optionally, you can add a passphrase to increase security.

• Resize Partition - if a customer upgrades a VM by changing the disk size to a larger one, the size of the last partition in that disk will be automatically increased by the newly added value.
  The feature deletes a current partition table, then creates a new one according to a new disk size, and reboots the VM.
  These partition types have to be standard, e.g. 'Linux Extended' (Ext2/Ext3/Ext4...), 'Linux SWAP' , 'Solaris' or 'LVM' .
  Important - in case of any failure, all data available on the resized partition might be lost!
• Change Root Password - this option is responsible for the automatic reset of the password after the VM creation. It also allows users to generate a new root password while using the VM.
• Delete VM SSH Root Key - it will automatically remove the default SSH key after the VM creation. Use it to increase security or leave unchecked for resizing partition feature.
  Important - if a customer decides to remove the public key from the VM on their own or the 'Delete VM SSH Root Key' option is enabled, it will make the 'Resize Partition' and 'Change Root Password' options no longer working!

From now on, all passwords for the newly created VMs will be generated automatically by WHMCS. The partition will be also automatically resized during the VM creation if a disk size is different than the one set as default in the template.
What is more, the administrator will be able to execute the action of partition resizing during the package change (if the disk size has been previously upgraded) and also set a custom root password in the 'Product/Services' settings.

You can monitor and manage each product from your WHMCS admin area.

You are also able to add an IP address to a product. To do so, press 'Add New IP' as shown on the following screen.

Note: The IP addresses you enter must be already existing in the IP Management pool and cannot be assigned to any other VM. This feature will also not work if you already use the IP Manager For WHMCS module.

RAM details:

Assigned - RAM assigned to the server (hosting services only)

Total - full server RAM

Free - difference between Total Server RAM and Assigned RAM

Suspended - number of suspended accounts per size in MB
i.e. (1/500 MB) when there is only one suspended account (occupying 500MB of that account) assigned to Proxmox VPS server.
You can quickly define VM ID ranges for those servers. To do so, press 'Manage' .
Note: VM ID range allows you to sort created VMs depending on the server they were created on.

Run 'recovery cron job' to create and store full backups of VMs configuration.
Use the 'Export to File' button to save information of the cron execution.

Please note that in order to use ISC DHCP, root details must be provided in the server configuration settings.

• Install DHCP package on your Proxmox server (if you are using more than one, then do it on each one of them). Execute the below command to do so:

sudo apt-get install isc-dhcp-server

• When ready, configure DHCP server in /etc/dhcp/dhcpd.conf file for IPv4 or /etc/dhcp/dhcpd6.conf for IPv6.
• Now, run the below command to create the file for IPv4:

sudo nano /etc/dhcp/dhcpd.conf

• Or the next command for IPv6 accordingly:

sudo nano /etc/dhcp/dhcpd6.conf

Below there is an exampple of how a properly created file should look like: /etc/dhcp/dhcpd.conf.
See what data it should contain:

# Sample /etc/dhcpd.conf
# (add your comments here) 
default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.example";
subnet 192.168.1.0 netmask 255.255.255.0 {
#ProxmoxIPv4
host 1 {hardware ethernet 32:62:39:64:33:33;fixed-address 192.168.1.2;}
host 2 {hardware ethernet 32:39:61:31:33:66;fixed-address 192.168.1.3;}
}

• Key words: #ProxmoxIPv6 #ProxmoxIPv4 suggesting the usage of a specific type of IP address must be located in one of the files mentioned above.
  

Note: Only a key word corresponding to the used IP type is required. If you do not use IPv6 then /etc/dhcp/dhcpd6.conf configuration and #ProxmoxIPv6 key word are not obligatory.

• More info about configuring DHCP server can be found here.

To use the automatic IP addresses assignment feature, select 'ISC DHCP Server' option in 'Module Settings' under the 'KVM Configuration' section.

By activating the IP Manager submodule, settings concerning IP addresses such as 'Configurable Options' and 'Default Options' will be ignored by Proxmox VPS.
Note: This integration works only for LXC and KVM virtualization type. OpenVZ is no longer supported.

1. Proceed to the IP Manager For WHMCS → Subnets tab in order to prepare IP addresses subnet for Proxmox VPS For WHMCS.
2. Edit the previously created IP addresses subnet and create a custom field 'MAC Address' with a desired MAC address at each IP address in the subnet.
Note: Setting up MAC address is optional and can be skipped.
More details about creating custom fields can be found here.

Note: Multiple IP addresses subnets can be assigned to a single configuration.
4. Now, link the configuration with Proxmox VPS product via a product, a server or a configurable option.
On the following screen we have linked them via the configurable option assigned to the Proxmox VPS product.

Press 'Activate' next to the 'proxmoxVPS' module type as shown on the following screen.

Press 'Configure' , choose a configuration and press 'Save Changes'.

In order to convert a VM, go to 'Proxmox Addon' → 'Templates' → 'Create Template'.
Choose a server on which the VM to convert exists and press 'Next Step'.
Important: VM will be destroyed in the process.

Fill out 'Template Friendly Name' and press 'Save Changes'.

Cron job user synchronization - if you change a user role in the product settings, this cron job changes the roles of existing users.
Cron job recovery VM synchronization - runs regularly, responsible for updating list of recovered VMs, allows downloading results.

These keys will be next used to enable the automatic resizing of a partition upon the disk upgrade as well as the change of a root password on the VM.
For more information, go to the Configuration of SSH Keys section.

Before you start, we recommend that you learn the details of cloud-init configuration on your Proxmox server first. Click here for more information.
Once you get familiar with Proxmox cloud-init, proceed to the Cloud-Init group of your KVM configuration where you can enable it and configure the following options:

• Service Username - if enabled, the username for the VM will be taken from the details of WHMCS product services. Otherwise, it will remain unchanged and set by default as in the template or taken from the 'Default User' field if provided.
• Service Password - if enabled, the password for the VM will be taken from the details of WHMCS product services. Otherwise, it will remain unchanged and set by default as in the template.
• Service Nameservers - if enabled, the nameservers for the VM will be taken from the details of WHMCS product services. Otherwise, they will remain unchanged and set by default as in the template.
• DNS Domain - provide a DNS domain for a container. During VM creation, the setting from the host will be automatically used if neither a search domain nor a nameserver is set.
• Default User - provide the username that will be used for VM instead of using the 'Service Username' field or the default user of the configured image. Otherwise, leave empty.

Additionally, you can use the following commands that can be useful for debugging:

• Re-run cloud init scripts

rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/* &&
cloud-init init  &&
cloud-init modules -m final

• Logs

cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log

As you can see, control panel contains options useful for managing a VM.
The module allows your customers to easily 'boot', 'reboot', 'stop' and 'shutdown' their servers through pressing specific buttons.
Clients can also access the 'noVNC' and 'SPICE' consoles. Among 'Additional Tools' you will find options to 'reinstall' server, create 'backup' and set up 'backup jobs'.

Your clients are able to access resource usage 'graphs', view VM 'history', manage 'network' interface device and set up 'firewall' rules.
Additionally, in case of Proxmox VPS (KVM) clients may take 'snapshots' and manage 'disks'.

Customers can also view 'DNS' (LXC only) and 'IP Addresses'.

To access it, simply press the 'VNC Console' button. The console requires Java software. You need to accept security risk to run it.
Please note that only versions of Proxmox prior to v.2.3.0 support this console.
Remember that VNC Console is available for users of Proxmox VE 3.4. only!

Note: Your client needs to download and install client side SPICE application in order to connect with a VM.

You can allow your clients to either select a new OS from the provided KVM templates to automatically replace the system, or let them choose the ISO image for manual system installation.

Note: The ISO images can only be used for the KVM virtualization type, while VM templates can be used either for KVM or LXC.
Additionally, the new OS password for the KVM OS reinstallation will be set based on the previous service password while for the LXC user can define the OS password before a new installation.

Afterward, press 'Add'.

There you can easily edit/delete and preview details of existing backup jobs.
To add a new backup job, press 'New Job' as shown on the following screen.

Next, press the 'Add' button.

They can also switch the timeline of graphs by choosing it from a dropdown menu pointed on the following screen. Choose from 'Hour', 'Day', 'Week', 'Moth' and 'Year'.

Keep in mind that ' Network device' options must be set to 'Private'.

As you can see, you can easily modify and remove the previously added rules and security groups.
To add a new firewall rule (1) or a predefined security group (2), use the buttons marked on the following screen.

The first way (red) is to choose macro from the 'Macro' dropdown field.
The second way (blue) requires you to choose a protocol and fill in both source and destination ports.
Additionally, you can enable a rule and add a comment to it.
Press 'Add' to create a rule.

Additionally, you can enable a security group and add a comment to it.
Click 'Add' to use the security group.

Go to 'Setup' → 'Products/Services' → 'Configurable Option' and press the edit button next to the configurable option used by your product.

Now, we will show you how to edit options. Let's say you want to add an option to order a VPS with 4 cores per socket.
To do so, edit 'The number of cores per socket' by pressing the edit button as shown on the screen below.

To add a new option, enter it in the textbox below all other options. If you want to replace the option you currently have, simply edit the textbox.
Finish by pressing 'Save Changes'.

The table below lists all configurable options used by the module for the chosen virtualization type.
Note that the name on the left is a variable (which you should not edit) separated by the | from the editable custom name.

The module allows you to charge your customers basing on the server resources used by them.
Note: Advanced Billing supports OpenVZ (prior to Proxmox v2.3.0), KVM and the LXC virtualization type.

1. Create and configure a Proxmox VPS server and a server group.
2. Create and configure a Proxmox VPS product.
3. Place a new order with a previously created product for your customer. Do not activate the product yet.
4. Access the just created product from the admin area via client profile → 'Products/Services' section and fill out 4 custom fields.
They are 'vmid' and 'node' .
5. If possible, enter IP addresses into fields 'Dedicated IP' and 'Assigned IPs' .
6. Set the product status to 'Active' .
Remember not to click the 'Create' button during the entire process. That would unnecessarily create a new virtual machine.

In order to allow defining locations for Proxmox VPS, follow the below steps:
1. In your product's 'Module Settings' set up 'Default Node' to 'Server-Node'.
2. Create a WHMCS server for each Proxmox node.
3. Manually create a configurable option, name it for example: 'Location'. Add options for each server within that config option.
4. Install our Server Allocator For WHMCS module and configure it properly.

It must assign a server to a product depending on the selected option in 'Location' configurable option.

Keep in mind that all locations must have exactly the same configuration.

Create action is triggered on the assigned server due to the 'Server-Node' option enabled.

When they are joined into one cluster, you can access all of them by connecting to just one.

• auto-node - Creates a new VM on the server (node) with the most free resources, so the administrator cannot control it.
  
• server-node - Creates a VM on the server node which IP address or node host (e.g. node1.yourproxmox.com) was provided during the WHMCS's server configuration,
  allowing the administrator to control the node choice.

In the product details view in admin area you can preview the current IP Sets on VM:

• Service Creation Failed - if there were any errors that prevented a VM from being successfully created, then the selected email template will be sent to the administrator
• Upgrade - select the email notification template you want to receive if the client requests the product upgrade.
  This will also block the automatic sending of an upgrade request to the Proxmox API so the administrator can review it first.
  Note that if chosen, the product will be still upgraded in the WHMCS itself but the upgrade request will not be passed to the API.
• To-Do List - if checked, in addition to sending any of the above notifications, it will also add them as tasks on the WHMCS To-Do List.

Please note that each time you update the module, you have to also update the files on the Proxmox server described in the step 4 of the installation guide.

Keep in mind there is a quick and easy option in our client area that will let you upgrade the license of your module to an open source version at any time - and at a lower price!
To take advantage of it, simply use the License Upgrade button located on the product's page and a discounted invoice will be generated automatically.

Reason: User account does not have permission to access the noVNC console.
Solution: Proceed to the product's 'Module Settings' tab and check whether selected a user permission grants access for the noVNC console.
Afterward, proceed to the client's hosting and run 'Change User Role' on it.

To change user role for all clients please use the 'Cron job users synchronization (run only once)' cron command.
You will find it in the 'Proxmox Addon' → 'General Settings'.

Reason: Your certificate is not valid.
Quick Solution: Your client needs to add the Proxmox address to the certificate exception list.
Best Solution: Configure a valid certificate.

Reason: You have not uploaded the additional files on the Proxmox server.
Solution: Open 'UploadOnProxmoxServer' and upload content of proper version on all Proxmox servers into the '/usr/share/novnc-pve/' folder.

Reason: You have not created a client yet.
Solution: When PAM authentication is used, it is necessary to create a client first in order to change the password correctly. Read more about it here.

Reason: You have not updated the php.ini file
Solution: After SSH2 PECL installation be sure that you have updated php.ini files for CLI and webserver.