Your customers will be able to single-handedly create as many servers as they need and manage aspects like backups, network devices or firewall.
The module supports Cloud-Init for the KVM virtualization type and features Proxmox Addon allowing you to easily manage servers, IP addresses and clusters.

We will guide you step by step through the whole installation and configuration process.

Note: You can check the current PHP version in your WHMCS. To do so, proceed to 'Utilities' › 'System' › 'PHP Info'.

The content of PHP version files should look like this.

Note: If you wish to allow noVNC consoles, you need to configure a product using the PVE realm and set user permissions to 'PVEVMUser.'

This file is located in 'modules/servers/proxmoxCloud/license_RENAME.php'. Rename it from 'license_RENAME.php' to 'license.php'.

Enter your license key between quotation marks as presented on the following screen.
You can find your license key in our client area → 'My Products'.

To do so, log in into your WHMCS and press 'Setup' → 'Addon Modules'. Find 'Proxmox Addon' and press 'Activate'.

Note: The Proxmox addon version might not match the current version of the provisioning module itself. This is completely normal due to the specific structure of the product.

To do so, click on the 'Configure' button. Tick 'Full Administrator' and press 'Save Changes'.

Go to 'Setup' → 'Products/Services' → 'Servers' and press 'Add New Server'.

Choose 'ProxmoxCloud' from a dropdown menu and press 'Save Changes'.
The last step is choosing authentication from the 'Authentication' dropdown menu.
Note: Authentication depends on the realm used by the user on a Proxmox server.

You can check connection with the Proxmox server. To do so, simply press 'Test Connection' as shown on the screen below.

Click on 'Create a New Group'.

To create a product, click on 'Create a New Product'.

'Product Type' can be set to 'Dedicated/VPS Server' but nameservers will have to be added later.
If you do not want to be forced to fill in nameservrs data, please select 'Hosting Account' instead.

Next, click on 'Save Changes'.

Available virtualization types:

• KVM
  
• LXC (supported by Proxmox VE 4.0 and later)
  
• OpenVZ (supported by Proxmox VE 3.4 and earlier, prior to Proxmox Cloud For WHMCS 2.3.0)
  

You can also choose how often the bandwidth usage should be calculated.
Afterwards, press 'Save Changes'.
Note: If you choose 'Auto-Node' as 'Default Note', new virtual servers will be created on the least used node.

From dropdown menus, choose 'KVM OS Type', 'Default Storage', 'Disk Type' and 'CD/DVD-ROM Type'.
Choose ISO images available for your clients by marking them in 'ISO Image'.

Set the network through selecting from the 'Network Mode', 'Network Model', and 'Bridge' dropdown menus.
If you are going to allow your clients to manage their private networks, you have to select 'Private Bridge' and set up VLAN TAG Range.
Type in your VM description in 'Container Description', set the backup location in the 'Storage' dropdown menu and click 'Save Changes'.
You may also decide whether you wish to use DHCP software to automatically assign IP addresses - if yes, simply select the option 'ISC DHCP Server'.
Read more about its installation here.

From dropdown menus, choose 'Default Storage', 'Template' and 'Bridge'.
Enter your virtual server description in 'Container Description' and press 'Save Changes'.

Remember! This virtualization type is supported only by Proxmox VE 4.0. and later.
If you are using the previous version of a Proxmox server, you will not be able to start this virtualization.

Please note that only Proxmox versions prior to v.2.3.0 support OpenVZ virtualization.

From dropdown menus, choose 'OpenVZ OS Templates', 'Default Storage', 'Network Mode' and 'Bridge'.
Type in your virtual server description in 'Container Description' and press 'Save Changes'.

Remember! This virtualization type is supported only by Proxmox VE 3.4 and previous.
If you are using the latest version of a Proxmox server, you will not be able to start this virtualization.

• Select 'Private Bridge'
• Select 'Private Network Model'
• Enter ranges for VLAN TAG
  

Note: Configure these three options for the KVM and LXC virtualization.
If you skip this step, clients will not be allowed to add private networks.

Define backup settings in the 'Backups' section through selecting values from the 'Storage' and 'Backups Routing' dropdown menus.
Backups routing automatically removes the oldest backup when a new backup is created and the established backup limit is reached.
Additionally, you can limit the time period for storing backups by providing the number of days into the 'Store The Backup For N Days' field.
Furthermore, you can define the limit of firewall rules and the interface to apply the rules to.
Finish the configuration in the 'Module Settings' tab through pressing 'Save Changes'.
Important: Keep in mind that the root password is generated automatically for the KVM virtualization upon its creation and reset.
If you want to be able to set your own password on the KVM virtualization, go check the Configuration of SSH Keys or the Cloud-Init section.

Press 'Generate default' next to 'Custom Fields'.
Static Resources: Fill in the fields in located in the 'Default Configuration' section with desired values.
Configurable Resources: Click on 'Generate Default' next to 'Configurable Options'.
Press 'Save Changes' at the bottom of the page.
Note: While generating 'Configurable Options', it is advisable to delete the ones you do not use, e.g. the swap option or IPv6 for the KVM virtualization.

php -q /your_whmcs/modules/servers/proxmoxCloud/cron/cron.php

Do not forget to replace 'your_whmcs' with your WHMCS root location.

In order to create a new product, you have to add a dedicated IP pool to 'IP Addresses' list in 'Proxmox' addon module.
To do so, go to 'Addons' → 'Proxmox Addon' → 'IP Management' and press 'New IP Addresses'.
Note: You can add both IPv4 and IPv6 addresses.

Select 'Private Address' if the added address is of such a kind, clients will be then allowed to add, remove and configure their private network.
Afterwards, press 'Add'.
Note: Automatic IP assignment works only for OpenVZ/LXC machines.
In case of the KVM virtualization, IP addresses have to be manually added to the pool in Proxmox addon or IP Manager to allow their automatic assignment.
Important: Private IP addresses are not downloaded for the IP Manager module.

If you used the KVM virtualization type, the client area of a successfully created product should look similar to the one presented on the screen below.

Allowed actions:

• Create
• Suspend/Unsuspend
• Terminate
• Change Package
• Change User Role

SSH keys are required to enable the automatic resizing of a partition upon the disk upgrade as well as the change of passwords for the KVM virtualization type.
Note - 'Resize Partition', and 'Change Root Password' will only work for newly created VMs with ISC DHCP and Internet access configuration.
We recommend using this option only for new products.

Then, generate SSH keys and add the public key to the root account on the newly created VM image. Moreover, upload both ( public and private ) keys on your WHMCS server in a secure location.

Optionally, you can add a passphrase to increase security.

• Resize Partition - if a customer upgrades a VM by changing the disk size to a larger one, the size of the last partition in that disk will be automatically increased by the newly added value.
  The feature deletes a current partition table, then creates a new one, according to a new disk size, and reboots the VM.
  These partition types have to be standard, e.g. 'Linux Extended' (Ext2/Ext3/Ext4...), 'Linux SWAP' , 'Solaris' or 'LVM' .
  Important - in case of any failure, all data available on the resized partition might be lost!
• Change Root Password - this option is responsible for the automatic reset of the password after the VM creation. It also allows users to generate a new root password while using the VM.
• Delete VM SSH Root Key - it will automatically remove the default SSH key after the VM creation. Use this component to increase security or leave unchecked for resizing partition feature.
  Important - if a customer decides to remove the public key from the VM on their own or the 'Delete VM SSH Root Key' is enabled, it will make the 'Resize Partition' and 'Change Root Password' options no longer working!

From now on, clients will be able to set their own password for newly created VMs.

At the same time, you can monitor and handle each product from your WHMCS admin area.
To view VM details(1), migrate VM(2), update VMs node(3) or delete VM(4), press a specific button.
To view details of a virtual server created by your client, simply press the 'Load Status' button.

Migration is useful when you want to request migration from the WHMCS admin area, while editing node is practical for updating details after migrating the virtual machine from the Proxmox panel.

RAM details:

Assigned - RAM assigned to the server (hosting services only)

Total - full RAM on server

Free - difference between Server Total RAM and RAM Assigned

Suspended - the number of suspended accounts per their size in MB
i.e. (1/500 MB) when there is only one suspended account (size: 500MB of that account) assigned to Proxmox server VPS.
You can quickly define VM ID ranges for those servers, press 'Manage' to do so.
Note: VM ID range allows you to sort the created virtual machines depending on the server they were created from.

Run 'recovery cron job' to create and store full backups of VM configuration.
Use the 'Export to File' button to save information of the cron execution.

Clients may manage their private networks and assign IP addresses to VM only if some private IP addresses are added.

In order to convert the virtual machine, go to 'Proxmox Addon' → 'Templates' → 'Create Template'.
Choose the server on which the virtual machine that should be converted into a template exists and press 'Next Step'.
Important: The virtual machine will be destroyed in the process.

Fill in 'Template Friendly Name' and press 'Save Changes'.

Cron job user synchronization - if you change the user role in the product settings, this cron job changes the roles of existing users.
Cron job recovery VM synchronization - run regularly, responsible for updating the list of recovered VMs, allows downloading results.

These keys will be next used to enable the automatic resizing of a partition upon the disk upgrade as well as the change of a root password on the VM.
For more information, go to the Configuration of SSH Keys section.

Note: Cloud-Init functionality is supported for the KVM virtualization only.

Important: These are only exemplary configuration scripts and may not work on your Proxmox server configuration!
Before using, please first adjust them to your needs and make sure they are configured correctly.

Here you can find some cloud config examples.

Before you start we recommend to learn the details about cloud-init configuration on your Proxmox server - click here for more details.
Once you get familiar with Proxmox cloud-init proceed to Cloud-Init group of your KVM configuration where you can enable it and set the following:

• Allow To Set Username - if enabled the client can set its own username during VM creation. Otherwise it will remain unchanged and be set by default as in the template or set from the 'Default User' field if provided.
• Allow To Set Password - if enabled the client can set its own password during VM creation Otherwise it will remain unchanged and be set by default as in the template.
• Allow To Set SSH Public Key - if enabled the client can upload its own public SSH key in OpenSSH format during VM creation. This will then allow client to log in to VM using their private key instead of password.
• Default User - provide the username that wille be used for VM instead of using the 'Service Username' field or image's configured default user. Othewise leave empty.
• Allow To Set DNS Domain - if enabled the client can provide its own DNS domain for a container. During VM creation, the provided setting will be automatically used if neither a search domain nor a nameserver is set.
• Allow To Set Nameservers - if enabled the client can set its own nameservers for the VM. Otherwise they will remain unchanged and be set by default as in the template.

Additionally you can use the following commands that can be useful for debugging.

• Re-run cloud init scripts

rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/* &&
cloud-init init  &&
cloud-init modules -m final

• Logs

cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log

They can also view resources used and manage existing virtual servers.

Instead of using the sliders you can also enter resources values into textboxes.
Confirm creation through pressing 'Add New Virtual Server'.
Note! Data to be filled in may differ according to which type OpenVZ or KVM you have chosen.

You can find information about managing your virtual servers in next section.

Please keep in mind that this option must be enabled by administrator in your product's 'Module Settings' previously and a private IP address must be added in the Proxmox Addon.
Otherwise you will no be able to add any private network successfully.

OS templates do not consume any resources such as CPU sockets, memory etc.
Important! When you convert server into a template, it is consumed in the process! There is no possibility to revert it back.

As you can see, control panel contains useful options needed for managing VPS.
The module allows your customers to easily boot, stop, reboot and shutdown their VM through pressing a specific button.
Clients can also upgrade server and access VNC, SPICE and noVNC consoles.
Any additional tools are described in details in below sections.

Console requires Java software and you need to accept security risk to run it.

Note: Your client needs to download and install client side SPICE application in order to connect with VM.

Afterwards, alter options you desire and confirm through pressing 'Save Changes' button.

You can provide your clients to either select a new OS from a provided KVM templates to automatically system replacement, or let them choose the ISO image for manual system installation.

Note: The ISO images can only be used for the KVM virtualization type, while VM templates can be used either for KVM or LXC.
Additionally the new OS password for the KVM OS reinstallation will be set based on the previous service password while for the LXC user can define it before the new installation.

Afterwards, press 'Add' button.

To add a new backup job press 'New Job' as shown on the following screen.

Next, press 'Add' button. You may also enable email notifications here.

They can also switch the timeline of the graphs by choosing it from a dropdown menu pointed on the following screen.

Additionally, each access to the VM through SPICE console is shown here.

Depending on the settings of the product (private network is used), they can also create/remove them. This feature is optional.

As you can see, you can easily modify and remove previously added rules and security groups.
To add a new firewall rule (1) or predefined security group (2) use the buttons marked on the following screen.

The first way (red) is by choosing macro from a 'Macro' dropdown field.
The second way (blue) consist of choosing protocol and filling source and destination ports out.
Additionally, you can enable rule and add a comment to it.
Press 'Add' to create a rule.

Additionally, you can enable security group and add a comment to it.
Click 'Add' to use the security group.

Important! This option is available only for 'KVM' virtualization type.
Read the below section to learn how to properly install and configure DHCP.

• Install DHCP server package on your Proxmox Node, execute the below command to do so:

sudo apt-get install isc-dhcp-server

• When ready, configure DHCP server in /etc/dhcp/dhcpd.conf file, run below command to create the file:

sudo nano  /etc/dhcp/dhcpd.conf

Below there is an exemplary content of properly created file: /etc/dhcp/dhcpd.conf.
Take a look at it to see what data should be inside of it:

# Sample /etc/dhcpd.conf
# (add your comments here) 
default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.example";
subnet 192.168.1.0 netmask 255.255.255.0 {
#ProxmoxIPv4
host 1 {hardware ethernet 32:62:39:64:33:33;fixed-address 192.168.1.2;}
host 2 {hardware ethernet 32:39:61:31:33:66;fixed-address 192.168.1.3;}
}

• Key words: #ProxmoxIPv6 #ProxmoxIPv4 suggesting the usage of a specific type of IP address, must be located in the above mentioned file.
  

Note: Only key word corresponding to a used IP type is required. If you do not use IPv6 then #ProxmoxIPv6 key word is not obligatory.

• More info about configuring DHCP server can be found here.

To use automatic IP addresses assignment feature, select 'ISC DHCP Server' option in 'Module Settings' under 'KVM Configuration' section.

Note: This integration works for LXC and KVM virtualization type. OpenVZ is no logner supported.

1. Proceed to IP Manager For WHMCS → Subnets tab in order to prepare IP addresses subnet for Proxmox Cloud For WHMCS.
2. Edit a previously created IP addresses subnet and create custom field 'MAC Address' with desired MAC address at each IP address in the subnet.
More about creating custom fields can be found here.

Note: Multiple IP addresses subnets can be assigned to a single configuration.
4. Now, link the configuration with Proxmox Cloud product via product, server or configurable option.
On the following screen we have linked them via a configurable option assigned to the Proxmox Cloud product.

Press 'Activate' next to 'proxmoxCloud' module type as shown on the following screen.

Press 'Configure', choose the configuration and press 'Save Changes'.

To do so, press 'Import VM' as shown on the following screen.

Confirm importing through pressing 'Save Changes' button.

Custom hooks are placed in 'your_whmcs/modules/servers/proxmoxCloud/customhooks.php' .
You can find there all custom hooks and basic information about them.
A full list of available custom hooks:

proxmoxCloud_beforeCreateVM 
proxmoxCloud_afterCreateVM
proxmoxCloud_beforeTerminateVM
proxmoxCloud_afterTerminateVM
proxmoxCloud_beforeCreate
proxmoxCloud_afterCreate
proxmoxCloud_beforeTerminate
proxmoxCloud_afterTerminate
proxmoxCloud_afterSuspend
proxmoxCloud_afterUnsuspend
proxmoxCloud_afterIPAdditionByAdmin
proxmoxCloud_afterIPDeletionByAdmin
proxmoxCloud_afterIPAdditionByUpgrade

Note! If automatically generated configurable options do not seem to fulfill your clients needs, you can edit them using this guide.

Module allows you to charge your customers basing on the server resources used by them.

To achieve that you need our Proxmox Cloud Autoscaling extension with Advanced Billing For WHMCS product.
Only then, when you have such a combination of products your offer will be full.

Advanced Billing For WHMCS will bill your clients for the currently used servers and its extension, Proxmox Cloud Autoscaling, will adjust the size of Cloud product to the current needs.

When they are joined into one cluster, you can access all of them by connecting to any of them.
While auto-node creates new VM on the server(node) with the most free resources, server-node creates VM on the server(node) which IP address it has (WHMCS's server configuration).

$_LANG['node_X']='Your friendly name for node_X';

Keep in mind there is a quick and easy option in our client area that will let you upgrade the license of your module to an open source version at any time - and at a lower price!
To take advantage of it, simply use the License Upgrade button located on the product's page and a discounted invoice will be generated automatically.

Reason: User account does not have permission to access noVNC console.
Solution: Proceed to product's 'Module Settings' tab and check whether selected user permission grant right for noVNC console.
Afterwards, proceed to client's hosting and run 'Change User Role' on it.

Reason: Your certificate is not valid.
Quick Solution: Your client needs to add Proxmox address to certificate exception list.
Best Solution: Configure valid certificate.

Reason: You have not uploaded the additional files on the Proxmox server.
Solution: Open 'UploadOnProxmoxServer' and upload content of proper version on all Proxmox servers into the '/usr/share/novnc-pve/' folder.

Reason: You have not updated php.ini file
Solution: After SSH2 PECL installation be sure that you have updated php.ini files for CLI and webserver.