Your customers will be able to manage virtual servers, backups, firewall, as well as access noVNC, SPICE, or Xterm.js console, and much more - all this directly in your WHMCS.
Additionally, the module provides support for Cloud-Init for the KVM virtualization type, and lets you conveniently control all existing servers, IP addresses and clusters through the built-in Proxmox Addon.

We will guide you step by step through the whole installation and configuration process.

Note: If you are still using any of the module's previous versions prior to v2.7.0, follow these instructions.

We will guide you step by step through the whole installation and configuration process.

In the most recent versions of the module, you will find only one package that supports PHP 7.1 and later.
If you need a package for the previous versions of PHP, please contact our support.

The first one that supports PHP 5.6 up to PHP 7.0, and the second one addressed to PHP 7.1 up to PHP 7.3.
Note: Keep in mind that PHP versions 5.6 up to 7.0 are no longer officially supported and their security issues are not fixed or released any more. Find more info here.

The content of the chosen PHP version files to upload should look like this.

Note: If you wish to allow noVNC consoles, you need to configure a product using the PVE realm and set user permissions to 'PVEVMUser.'
Important: Open Folder UploadOnProxmoxServer/5.2-10 and upload content on all your proxmox servers. Scripts must be updated on proxmox server v2.6.0 to allow Xterm.js console work.

The file is located in 'modules/servers/proxmoxVPS/license_RENAME.php'. Rename it from 'license_RENAME.php' to 'license.php'.

Enter your license key between the quotation marks as presented on the following screen.
You can find your license key in our client area → 'My Products'.

To do so, log in to your WHMCS and press 'Setup' → 'Addon Modules'. Find 'Proxmox Addon' and press 'Activate'.

Note: The Proxmox addon version might not match the current version of the provisioning module itself. This is completely normal due to the specific structure of the module.

To do so, click on the 'Configure' button, tick 'Full Administrator' and press 'Save Changes'.

Go to 'Setup' → 'Products/Services' → 'Servers' and press the 'Add New Server' button.

Enter the and password, choose the authentication type from 'Authentication' and 'Proxmox VPS' from the 'Type' dropdown menus. Save changes afterward.

Click on 'Create a New Group'.

To create a product, click on 'Create a New Product'.

'Product Type' can be set to 'Server/VPS' but nameservers will need to be added later (here you can find instructions on how to remove them from your order template).
'Server/VPS' type can also be useful for a more complex display of IP addresses assignment. However, if this is not something you need, you can select 'Shared Account' .

The page should reload automatically after the selection. If it doesn't, you may need to click 'Save Changes' .

Available virtualization types:

• KVM - short form for Kernel-based virtual machines; used for Qemu (Quick Emulator) which is an open source hypervisor that emulates a physical computer
• LXC - used for Linux Container as a lightweight alternative to fully virtualized virtual machines

Remember that changing the node will always result in the replacement of the current configuration with the default one. Upon selecting the node, the current configuration should be saved automatically.
If, for any reason, this does not happen, please press 'Save Changes'.

Note: If you choose the 'Auto-Node' as your 'Default Node', new virtual servers will be created on the least used node.
Note 2: An advanced tool is available for you to automatically manage the node selection with the best amount of resources. Read more about Load Balancer.

Before you start configuring the KVM virtualization, we highly recommend that you familiarize yourself with the following 'Qemu/KVM Virtual Machines' Proxmox wiki article.

Since the 2.7.0 module version, the basic fields required to create a virtual machine are filled in by default to make the initial setup simpler for you.
Additionally, only the fields that require the administrator's initial attention are expanded by default, while all panel sections with advanced settings are collapsed.

Start your product configuration in the 'Virtual Machine Configuration' panel, where you can set up such features as the used OS template, machine resources and limits or assigned IP addresses.
Note that some fields can be left empty if you want to disable them or set as unlimited - read the tooltips for further guidance.

Other panels that are less important or require more knowledge can be expanded according to your needs by clicking on the arrow on the right.

Save settings when ready.

If you want to set your own password on the KVM virtualization, use the Cloud-Init feature.

Before you start configuring the LXC virtualization, we highly recommend that you familiarize yourself with the following 'Linux Container' Proxmox wiki article.

Just as with KVM, the basic fields required to create a virtual machine are filled in by default since the 2.7.0 module version to make the initial setup simpler for you.
Additionally, only the fields that require the administrator's initial attention are expanded by default, while all panel sections with advanced settings are collapsed.

Start your product configuration in the 'Virtual Machine Configuration' panel, where you can set up such features as 'Default Storage', used 'OS Template', container resources and limits or the number of assigned IP addresses.
Note that some fields can be left empty if you want to disable them or set as unlimited - read the tooltips for more information.

Other panels that are less important or require more knowledge can be expanded according to your needs by clicking on the arrow on the right.

Save settings when ready.

This step is optional - if you want to take it, start by selecting which of the configurable options listed in this table you want to generate.

Note: it is advisable not to generate options that you do not plan to use, or in case you want their fixed values to be taken from the module settings, rather than being flexible.
Moreover, the configurable options differs between the KVM and LXC virtualization types, so create them separately if needed.

More information about configurable options and their edition can be found here.

php -q /your_whmcs/modules/addons/proxmoxAddon/cron/cron.php queue

Do not forget to replace 'your_whmcs' with your WHMCS root location. The cron can be also executed every single minute in order to perform the action more often.
In the Configuration of Addon settings, you will find the personalized URL for the cron on your server as well as more possible crons to set up for execution.

Please note there are separate cron jobs dedicated to Proxmox VPS and Proxmox Cloud, crucial for the seamless operation of the modules. Their suggested frequency is 5 minutes.
If you use only one of the Proxmox products, there is no need to set up separate cron jobs.
The other cron jobs are responsible for:

• Users synchronization (run once only)
• VM recovery synchronization (every 5 minutes suggested)
• VM migration synchronization (every 5 minutes suggested)
• Tasks synchronization (every 5 minutes suggested)

In order to create a new product with an IP address assigned, you have to add 'Dedicated IP ' to the 'IP Addresses' list in 'Proxmox' addon module.
To do so, go to 'IP Management' and press 'New IP Addresses'.
Note: You can add both the IPv4 and IPv6 addresses.

Then adjust the mask that will be used to determine the number of generated addresses - remember that different values are used for creation of the same number of IPv4 and IPv6 subnets.
Next, choose a server from the 'Server' dropdown menu and define which virtualization it should be applied to through selecting it from the 'Virtualization' dropdown menu.
Afterward, press 'Confirm'.

If you are using the KVM virtualization type, the client area with a successfully created product should look similar to the one on the screen below.
The assigned IP addresses should be displayed on the bottom panel according to the way you configured the IP pools. It looks quite similar in the case of the LXC virtualization type product.

Note: Cloud-Init functionality is supported for the KVM virtualization only.

Before you start, we highly recommend that you learn the details of Cloud-Init configuration on your Proxmox server first. Click here for more information included directly in the Proxmox wiki article.
Once you get familiar with Proxmox Cloud-Init, proceed to the Cloud-Init group of your KVM configuration where you can enable it as well as configure the following options:

• Service Username - if enabled, the username for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, it will remain unchanged and set by default as in the template or taken from the 'Default User' field if provided.
• Service Password - if enabled, the password for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, it will remain unchanged and set by default as in the template.
• Service Nameservers - if enabled, the nameservers for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, they will remain unchanged and set by default as in the template.
• DNS Domain - provide a DNS domain for the container. During VM creation, the setting from the host will be automatically used if neither a search domain nor a nameserver is set.
• Default User - provide the username that will be used for the VM instead of using the 'Service Username' field or the default user of the configured image. Otherwise, leave empty.
• Custom - here you can optionally specify your custom config files that will be used instead of the automatically generated ones.

Additionally, you can use the following commands that can be useful for debugging:

• Re-run cloud init scripts

rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/* &&
cloud-init init  &&
cloud-init modules -m final

• Logs

cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log

You can monitor and manage each product from your WHMCS admin area.

1. At the beginning, there are standard module commands that you can perform on the VM.
2. There you will find actions to manage the state of the machine, access to remote consoles and its migration between nodes.
3. At this panel you will also find a preview of the basic information about the VM
4. And here you can change VM hostname, ISO image, bootable devices as well as insert SSH Public Key

This option must be previously enabled in the product configuration page and successfully installed on the VM. If so it should look similar to the image below.

Note: The IP addresses you enter must already exist in the IP Management pool and cannot be assigned to any other VM. This feature will also not work if you already use the IP Manager For WHMCS module.

RAM details:

Assigned - RAM assigned to the server (hosting services only)

Total - full server RAM

Free - difference between Total Server RAM and Assigned RAM

Suspended - number of suspended accounts per size in MB E.g.: (1/500 MB) when there is only one suspended account (occupying 500MB of that account) assigned to Proxmox VPS server.

Press 'More Details' icon and you will get extended information on the selected server. See below for more details.

To view information about a VM, simply press the more info icon.

In order to convert a VM, press 'Create Template' button.

Fill out 'Description' field and press 'confirm'.
Important: Selected VM will be destroyed in the process.

Note: VM ID range allows you to sort created VMs depending on the server they were created on..

Run the 'VM recovery synchronization' cron job to create and store full backups of VMs configuration, you can find it at point no 26 of the configuration instructions.
Use the 'Export to File' button to save information in a TXT copy file.

Run the 'Tasks synchronization' cron job to synchronize the tasks and view all tasks from the server, you can find it at point no 26 of the configuration instructions.

• private/public address
• servers or nodes that will use them
• virtualization types (different VMs will use different IP pools)

By activating the IP Manager submodule, settings concerning IP addresses such as 'Configurable Options' and 'Default Options' will be ignored by Proxmox VPS.
Note: This integration works for LXC and KVM virtualization types.

1. Proceed to the IP Manager For WHMCS → Subnets tab in order to prepare IP addresses subnet for Proxmox VPS For WHMCS.
2. Edit the previously created IP addresses subnet and create a custom field 'MAC Address' with a desired MAC address at each IP address in the subnet.
Note: Setting up MAC address is optional and can be skipped.
More details about creating custom fields can be found here.

4. Now, link the configuration with Proxmox VPS product via a product, a server or a configurable option.
On the following screen we have linked them via the configurable option assigned to the Proxmox VPS product where each of fields represent the number of assigned IP addresses.
You can of course mix them to your needs, where e.g. a single config option will assign provided numbers for both IPv4 and IPv6 addresses.

Press 'Activate' next to the 'proxmoxVPS' module type as shown on the following screen.

Press 'Configure' , choose a configuration and press 'Save Changes'.

Look above for the detailed instruction on how to proceed, follow steps 1-6 to create the new custom field.
The only thing you need to change is the custom field name, which should be 'Tag' in this case.

Set the weight of the resource that will prioritize its selection when sorting the Nodes according to the free resources.
Note: The lowest possible weight is 1. You cannot set for example 0,5.

Define the behavior of the tool:

• On Upgrade - select action to perform:
  • None - if selected, no action will be triggered
  • Block - the upgrade process will blocked as there are insufficient resources on the node
  • Migrate - the upgrade process will be performed after previous migration of the VM to a server with sufficient free space
• Shutdown VM on Upgrade
• Stop VM If Shutdown Fails

Let's take exemplary weights for each resource just like presented on the screen above.

VMs Number Weight: 1,5

CPU Weight: 2 

Disk Weight: 4

RAM Weight: 2,5 

Disk Weight with the highest weight which means it has the highest priority and VMs Number Weight as the least crucial factor on a given node.

In our situation, there are four nodes created, named A, B, C, D with current resources as shown below:

Node A (Free Ram: 60%, Free CPU: 20%, Free Disk Space: 50%, Number of VMs: 5) 

Node B (Free Ram: 30%, Free CPU: 70%, Free Disk Space: 40%, Number of VMs: 10) 

Node C (Free Ram: 90%, Free CPU: 50%, Free Disk Space: 20%, Number of VMs: 20) 

Node D (Free Ram: 80%, Free CPU: 60%, Free Disk Space: 70%, Number of VMs: 15)

The next step is sorting the nodes according to each resource, by assigning them corresponding priorities from 1 to 4.
The higher the priority, the more free resources available.

Sorting nodes is based on the following calculation method:

Final Node Priority = (RAM Priority x RAM Weight) + (CPU Priority x CPU Weight) + (Free Disk Priority x Disk Weight) + (No. VMs  Priority x VMs Weight)

The results for nodes A, B, C and D (with weights and sorting priority) are following:

Node A: (2 x 2,5) + ( 1 x 2 ) + ( 3 x 4 ) + ( 4 x 1,5 ) = 25
Node B: (1 x 2,5) + ( 4 x 2 ) + ( 2 x 4 ) + ( 3 x 1,5 ) = 23
Node C: (4 x 2,5) + ( 2 x 2 ) + ( 1 x 4 ) + ( 1 x 1,5 ) = 19,5
Node D: (3 x 2,5) + ( 3 x 2 ) + ( 4 x 4 ) + ( 2 x 1,5 ) = 32,5

As you can see, Node D has got the highest priority value with a result: 32,5.
It will be selected by Load Balancer to create a new VM there, to migrate or upgrade an existing VM if its current node does not have sufficient resources.

The above described calculations are performed every time, a 'Create' or 'Upgrade' actions are executed based on the current usage of resources on available nodes.

Important: In case a given client already has an existing VM on a node, this very node will NOT be taken into account during the prioritization of resources process.
This is due to security measures to make sure, that in case of failure of this node, the client will still have access to the other VM server, created on another, working node.
However, if this client has VMs on every node already, all of them will be taken into account during the selection prioritization process again.

If you decide to use Load Balancer tool, define the resources weights to adjust their prioritization of nodes according to your needs.

Note: The logs will be displayed only in case the errors occurred. If task finished successfully there will be nothing to see there.

As you can see, control panel contains options useful for managing a VM.
The module allows your customers to easily 'start', 'reboot', 'stop' and 'shutdown' their servers through pressing specific buttons.
Clients can also access the 'noVNC' , 'SPICE' and 'Xterm.js' consoles. Among 'Additional Tools' you will find options to 'reinstall' server, create 'backup' or set up 'backup jobs'.

Your clients are able to access resource usage 'graphs', view VM 'history', manage 'network' interface device and set up 'firewall' rules. Additionally, clients may take 'snapshots' and manage 'disks'.
On the main product page in the client area, under the 'Information' headline your clients can find information like about current usage of CPU, memory, disk (LXC only) and SWAP (LXC only).
Customers can also view 'DNS' (LXC only) as well ass assigned 'IP Addresses'.

Note: Bellow example image was taken from the KVM based VM, but for LXC it will looks very similar.

Note: Since 2.7.0 version of the module your clients no longer need to download and install client side SPICE application in order to connect with a VM. The console will be now opened in the virt-viewer instead.

You must update the scripts on Proxmox v2.6.0. To do so open folder: UploadOnProxmoxServer/5.2-10 and upload content on all your Proxmox servers.

You can allow your clients to either select a new OS from the provided KVM templates to automatically replace the system, or let them choose the ISO image for manual system installation.

Note: The ISO images can only be used for the KVM virtualization type, while VM templates can be used either for KVM or LXC.
Additionally, both KVM and LXC users can define the new OS password before a new installation.

There you can easily edit/delete and preview details of existing backup jobs.
To add a new backup job, press 'New Job' as shown on the following screen.

Finally, press the 'Confirm' button.

They can also switch the timeline of graphs by choosing it from a dropdown menu pointed on the following screen. Choose from 'Hour', 'Day', 'Week', 'Moth' and 'Year'.

If an error has occurred during the execution of a task, the client can view API response error by clicking on the 'Information' icon.

Keep in mind that ' Network device' options must be set to 'Private'.
Also, after network creation, the button to add another one or delete a newly created private network may not be longer available, depending on your Proxmox configuration.

As you can see, you can easily modify and remove the previously added rules and security groups.
To add a new firewall rule (red arrow) or a predefined security group (blue arrow), use the buttons marked on the following screen.

The first way (orange) is to choose macro from the 'Macro' dropdown field.
The second way (blue) requires you to choose a protocol and fill in both source and destination ports.
Additionally, you can enable a rule and add a comment to it.
Press 'Add' to create a rule.

Click 'Add' to use the security group.

Go to 'Setup' → 'Products/Services' → 'Configurable Option' and press the edit button next to the configurable option used by your product.

Now, we will show you how to edit options. Let's say you want to add an option to order a VPS with 4 cores per socket.
To do so, edit 'The number of cores per socket' by pressing the edit button as shown on the screen below.

To add a new option, enter it in the textbox below all other options. If you want to replace the option you currently have, simply edit the textbox.
Finish by pressing 'Save Changes'.

The table below lists all configurable options used by the module for the chosen virtualization type.
Note that the name on the left is a variable (which you should not edit) separated by the | from the editable custom name.

The module allows you to charge your customers basing on the server resources used by them.
Note: Advanced Billing supports OpenVZ (prior to Proxmox v2.3.0), KVM and the LXC virtualization type.

1. Create and configure a Proxmox VPS server and a server group.
2. Create and configure a Proxmox VPS product.
3. Place a new order with a previously created product for your customer. Do not activate the product yet.
4. Access the just created product from the admin area via client profile → 'Products/Services' section and fill out 4 custom fields.
They are 'vmid' and 'node' .
5. If possible, enter IP addresses into fields 'Dedicated IP' and 'Assigned IPs' .
6. Set the product status to 'Active' .
Remember not to click the 'Create' button during the entire process. That would unnecessarily create a new virtual machine.

In order to allow defining locations for Proxmox VPS, follow the below steps:
1. In your product's 'Module Settings' set up 'Default Node' to 'Server-Node'.
2. Create a WHMCS server for each Proxmox node.
3. Manually create a configurable option, name it for example: 'Location'. Add options for each server within that config option.
4. Install our Server Allocator For WHMCS module and configure it properly.

It must assign a server to a product depending on the selected option in 'Location' configurable option.

Keep in mind that all locations must have exactly the same configuration.

Create action is triggered on the assigned server due to the 'Server-Node' option enabled.

When they are joined into one cluster, you can access all of them by connecting to just one.

• auto-node - Creates a new VM on the server (node) with the most free resources, so the administrator cannot control it.
  
• server-node - Creates a VM on the server node which IP address or node host (e.g. node1.yourproxmox.com) was provided during the WHMCS's server configuration,
  allowing the administrator to control the node choice.

In the product details view in admin area you can preview the current IP Sets on VM:

• Service Creation Failed - if there were any errors that prevented a VM from being successfully created, then the selected email template will be sent to the administrator
• Upgrade - select the email notification template you want to receive if the client requests the product upgrade.
  This will also block the automatic sending of an upgrade request to the Proxmox API so the administrator can review it first.
  Note that if chosen, the product will be still upgraded in the WHMCS itself but the upgrade request will not be passed to the API.
  To finalize the upgrade use the 'Change Package' module command in your admin area.
• To-Do List - if checked, in addition to sending any of the above notifications, it will also add them as tasks on the WHMCS To-Do List.

Please note that each time you update the module, you have to also update the files on the Proxmox server described in the step 4 of the installation guide.

Important: After upgrading the module to the newest version please first enter to the Proxmox addon in your WHMCS before setting up the product or opening it from admin or client area!
Otherwise, when you try to open the product's module settings, you may get SQLSTATE" "Base table or view not found" errors.

Please also note that since 2.7.0 module version the commands for the crons execution have slighty changed - although the old ones should still work, we recommend that you update their entries.

Keep in mind there is a quick and easy option in our client area that will let you upgrade the license of your module to an open source version at any time - and at a lower price!
To take advantage of it, simply use the License Upgrade button located on the product's page and a discounted invoice will be generated automatically.

Reason: User account does not have permission to access the noVNC console.
Solution: Proceed to the product's 'Module Settings' tab and check whether selected a user permission grants access for the noVNC console.
Afterward, proceed to the client's hosting and run 'Change User Role' on it.

To change user role for all clients please use the 'Cron job users synchronization (run only once)' cron command.
You will find it in the 'Proxmox Addon' → 'General Settings'.

Reason: Your certificate is not valid.
Quick Solution: Your client needs to add the Proxmox address to the certificate exception list.
Best Solution: Configure a valid certificate.

Reason: You have not uploaded the additional files on the Proxmox server.
Solution: Open 'UploadOnProxmoxServer' and upload content of proper version on all Proxmox servers into the '/usr/share/novnc-pve/' folder.

Reason: You have not created a client yet.
Solution: When PAM authentication is used, it is necessary to create a client first in order to change the password correctly. Read more about it here.

Reason: You have not updated the php.ini file
Solution: After SSH2 PECL installation be sure that you have updated php.ini files for CLI and webserver.

Solution: Turn off the Xterms.js console in the product configuration.

Reason: User is other than "root" or server authentication is set to Proxmox VE
Solution: In server configuration, set the root details and select PAM as the authentication type.