Your customers will be able to manage virtual servers, backups, firewall, as well as access noVNC, SPICE, or Xterm.js console, and much more - all this directly in your WHMCS.
Additionally, the module provides support for Cloud-Init for the KVM virtualization type, and lets you conveniently control all existing servers, IP addresses and clusters through the built-in Proxmox Addon.

ModulesGarden is official Proxmox Solution Partner and this module is recommended by Proxmox Server Solutions GmbH.

}{|

We will guide you step by step through the whole installation and configuration process.

Note: If you are still using any of the module's previous versions prior to v2.7.0, follow these instructions.

We will guide you step by step through the whole installation and configuration process.

The first one that supports PHP 7.3 up to PHP 7.4, and the second one addressed to PHP 8.1.
Note: Keep in mind that PHP versions 5.6 up to 7.2 are no longer officially supported and their security issues are not fixed or released any more. Find more info here.

The content of the package to upload should look like this.

Note 1: If the version of the Proxmox server you are using is not available, choose the package with the highest version number that is lower than the one you are currently using.
Note 2: If you wish to allow noVNC consoles, you need to configure a product using the PVE realm and set user permissions to 'PVEVMUser.'
Important: Open Folder UploadOnProxmoxServer/5.2-10 and upload content on all your proxmox servers. Scripts must be updated on proxmox server v2.6.0 to allow Xterm.js console work.

The file is located in 'modules/servers/proxmoxVPS/license_RENAME.php'. Rename it from 'license_RENAME.php' to 'license.php'.

Enter your license key between the quotation marks as presented on the following screen.
You can find your license key in our client area → 'My Products'.

This folder is available at two locations:

1. your_whmcs/modules/addons/proxmoxAddon/
2. your_whmcs/modules/servers/proxmoxVPS/

• delete the "ProxmoxAddon_Backups" table from the database manually, then deactivate and activate the module in 'Addon Modules'
• enter the below command in phpMyAdmin:

ALTER TABLE `ProxmoxAddon_Backups` ADD `format` VARCHAR(40) NOT NULL AFTER `volid`;

To do so, log in to your WHMCS and navigate to 'System Settings' → 'Addon Modules'. Find 'Proxmox Addon' and press 'Activate'.

Note: The Proxmox addon version might not match the current version of the provisioning module itself. This is completely normal due to the specific structure of the module.

To do so, click on the 'Configure' button, tick 'Full Administrator' and press 'Save Changes'.

Go to 'System Settings' → 'Servers' and press the 'Add New Server' button.

Enter the and password, choose the authentication type from 'Authentication' and 'Proxmox VE VPS' from the 'Type' dropdown menus. Save changes afterward.

Click on 'Create a New Group'.

To create a product, click on 'Create a New Product'.

'Product Type' can be set to 'Server/VPS' but nameservers will need to be added later (here you can find instructions on how to remove them from your order template).
'Server/VPS' type can also be useful for a more complex display of IP addresses assignment. However, if this is not something you need, you can select 'Shared Account' .

The page should reload automatically after the selection. If it doesn't, you may need to click 'Save Changes' .

Available virtualization types:

• KVM - short form for Kernel-based virtual machines; used for Qemu (Quick Emulator) which is an open source hypervisor that emulates a physical computer
• LXC - used for Linux Container as a lightweight alternative to fully virtualized virtual machines

Remember that changing the node will always result in the replacement of the current configuration with the default one. Upon selecting the node, the current configuration should be saved automatically.
If, for any reason, this does not happen, please press 'Save Changes'.

Note: If you choose the 'Auto-Node' as your 'Default Node', new virtual servers will be created on the least used node.
Note 2: An advanced tool is available for you to automatically manage the node selection with the best amount of resources. Read more about Load Balancer.

Before you start configuring the KVM virtualization, we highly recommend that you familiarize yourself with the following 'Qemu/KVM Virtual Machines' Proxmox Virtual Environment wiki article.

Since the 2.7.0 module version, the basic fields required to create a virtual machine are filled in by default to make the initial setup simpler for you.
Additionally, only the fields that require the administrator's initial attention are expanded by default, while all panel sections with advanced settings are collapsed.

Start your product configuration in the 'Virtual Machine Configuration' panel, where you can set up such features as the used OS Template or Archive, machine resources and limits or assigned IP addresses.
Note that some fields can be left empty if you want to disable them or set as unlimited - read the tooltips for further guidance.

Note: If you want to create the VM based on 'Archive' of the backup copy, remember to leave the 'OS Template' field empty.
Note 2: To turn off loading archives, see section: Tips, point 26.

Guest Agent is used to exchange information between the virtualization server and the operating system of your virtual machine.
Thanks to this, additional commands such as setting IP addresses and the user’s password can be executed in the guest system.
The 'Service Password' and 'Configure Network' options work with Windows OS only.
If enabled, while creating a VM, Guest Agent will set up the user’s password and configure the network automatically.
Remember that you need to install Guest Agent on your KVM template in order for the function to work properly. For more info click here

Save settings when ready.

If you want to set your own password on the KVM virtualization, use the Cloud-Init feature.

Before you start configuring the LXC virtualization, we highly recommend that you familiarize yourself with the following 'Linux Container' Proxmox wiki article.

Just as with KVM, the basic fields required to create a virtual machine are filled in by default since the 2.7.0 module version to make the initial setup simpler for you.
Additionally, only the fields that require the administrator's initial attention are expanded by default, while all panel sections with advanced settings are collapsed.

Start your product configuration in the 'Virtual Machine Configuration' panel, where you can set up such features as 'Default Storage', used 'OS Template' or 'Archive', container resources and limits or the number of assigned IP addresses.
Note that some fields can be left empty if you want to disable them or set as unlimited - read the tooltips for more information.

Note: If you want to create the VM based on 'Archive' of the backup copy, remember to leave the 'OS Template' field empty.

Other panels that are less important or require more knowledge can be expanded according to your needs by clicking on the arrow on the right.

Save settings when ready.

This step is optional - if you want to take it, start by selecting which of the configurable options listed in this table you want to generate.

Note: it is advisable not to generate options that you do not plan to use, or in case you want their fixed values to be taken from the module settings, rather than being flexible.
Moreover, the configurable options differs between the KVM and LXC virtualization types, so create them separately if needed.

More information about configurable options and their edition can be found here.

php -q /your_whmcs/modules/addons/proxmoxAddon/cron/cron.php queue

Do not forget to replace 'your_whmcs' with your WHMCS root location. The cron can be also executed every single minute in order to perform the action more often.
In the Configuration of Addon settings, you will find the personalized URL for the cron on your server as well as more possible crons to set up for execution.

Please note there are separate cron jobs dedicated to Proxmox VPS and Proxmox Cloud VPS, crucial for the seamless operation of the modules. Their suggested frequency is 5 minutes.
If you use only one of the Proxmox products, there is no need to set up separate cron jobs.
The other cron jobs are responsible for:

• Backups synchronization (every 5 minutes suggested) Important: required since version 3.7.0!
• Users synchronization (run once only)
• VM recovery synchronization (every 5 minutes suggested)
• VM migration synchronization (every 5 minutes suggested)
• Tasks synchronization (every 5 minutes suggested)
• VM power tasks (optional, each 5 minutes suggested)
• Resources notifications (optional, each 5 minutes suggested)
• Updating the server usage (each 5 minutes suggested)
• Removing backups (each 24 hours suggested)
• Snapshot jobs (optional, each 5 minutes suggested)
• Removing snapshots (optional, each 24 hours suggested)
• VM names synchronization (run only once)

Here, you may also turn on the 'Debug Mode' if you need to register module logs.

In order to create a new product with an IP address assigned, you have to add 'Dedicated IP ' to the 'IP Addresses' list in 'Proxmox' addon module.
To do so, go to 'IP Management' and press 'New IP Addresses'.
Note: You can add IPv4, IPv6 addresses and IPv6 subnets. The process is the same for all types.

Then, depending on the selected type, provide:

For IPv4 and IPv6 addresses types

• "Network Address" (eg: 192.168.1.0/24)
• "Gateway"
• "First IP Address" and "Last IP Address" - defines the border from which and up to which of the IP address the generation (including the provided addresses)

For "IPv6 Subnet" address type

• "Network Address" (eg: 192.168.1.0/24)
• "Gateway"
• 'Number of Subnets"
• "CIDR"
• "First IP Address" and "Last IP Address" - defines the border from which and up to which of the IP address the generation (including the provided addresses)

Decide whether the created addresses will be private or not.

Remember to press 'Confirm' button when ready.

If you are using the KVM virtualization type, the client area with a successfully created product should look similar to the one on the screen below.
The assigned IP addresses should be displayed on the bottom panel according to the way you configured the IP pools. It looks quite similar in the case of the LXC virtualization type product.

To work properly, proxy must be accessible from both public and private net as clients connect with proxy from the public net, without the need to connect directly to proxmox.

The file can be found under the following link after you have logged in to your ModulesGarden account.

It is advised to use the KVM virtualization type.

Warning! The used VPS may NOT have any services running, such as: PHP, WebServer (e.g. apache, nginx) or MySQL server.

ping modulesgarden.com

If everything is fine, assure that basic server security settings have been established.
You may turn off login with a password and turn on login with a key, change the SSH port, set up firewalls etc.

https://docs.docker.com/engine/install/
https://docs.docker.com/compose/install/

Then unzip the files.

cd /root/proxmox-proxy  

Note that the exact directory depends on the previous step where you uploaded the files.

Then open the just renamed '.env' file and provide a random set of signs to create the database and root passwords: DB_PASSWORD and MYSQL_ROOT_PASSWORD
These data will be automatically used when creating the database.

docker-compose up --build -d

docker exec -it proxyphp bash 

and run the installation script:

php -q /var/www/html/installer.php install 

When ready, find the API Key and keep it safe, you will need it later on when pasting the data into the Proxmox Addon, see point 12 for details.

ip a

Note: Remember to check the IP address in your virtual machine, not in the proxyphp conatiner.

Move to 'Addons' → 'Proxmox Addon' → 'Settings'. At the bottom you will find section 'Console Proxy' with 'Host' and 'API Key' fields.
Note that the Host field may include either an IP address or a domain. It is extremely important that you paste here the same data as the ones used in the SSL certificate.
The below screen shows an example with an IP address which is not the only correct possible option here.

Provide the necessary data and confirm the changes.

Remember to replace {vmip} with a valid virtual machine IP address, for example: https://192.168.8.113:8443/
When you see {"errors":["Invalid API Endpoint"]} it means that the server correctly responds.

• move to the '/root/proxmox-proxy catalog (the directory may differ depending on your setup)
• replace the nginx/localhost.crt and nginx/localhost.key files
• run the following command:

docker-compose up --build -d

Note: Cloud-Init functionality is supported for the KVM virtualization only.

Press 'Add New' to manually add your script.

• Username
• Hostname
• Password

Here is an exemplary configuration of a Cloud-Init script. You may alter it to fit your settings and paste into the 'Script' field:

#cloud-config
hostname: {$domain}
manage_etc_hosts: true
fqdn: {$domain}
user: {$username}
password: {$passwordHash}
chpasswd:
  expire: False
users:
  - default
package_upgrade: true

{$userid} {$serviceid} {$domain} {$username} {$password} {$passwordHash} {$model->dedicatedip} {$model->assignedips} {$customfields.sshkeys}

Where:

• $model variables - stands for following services
• $customfields variable - stands for all custom fields that exist in your service
  

Important: If you want to use a {} sign in your script that has no variable inside, you must place it inside: {literal}{/literal}, for example: '{literal}{"X": "Y"}{/literal}'.
Otherwise, errors will occur.

Move to Servers → Servers List → Details → Settings, there you will find fields that are responsible for the correct SSH connection.

• SSH Host
• SSH Port
• SSH User
• SSH Password or SSH Private Key
  

You also have to select the 'Snippet Storage' to locate the Cloud-Init script in the correct catalog.

Before you start, we highly recommend that you learn the details of Cloud-Init configuration on your Proxmox server first. Click here for more information included directly in the Proxmox wiki article.
Once you get familiar with Proxmox Cloud-Init, proceed to the Cloud-Init group of your KVM configuration where you can enable it as well as configure the following options:

• Service Username - if enabled, the username for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, it will remain unchanged and set by default as in the template or taken from the 'Default User' field if provided.
• Service Password - if enabled, the password for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, it will remain unchanged and set by default as in the template.
• Service Nameservers - if enabled, the nameservers for the VM will be taken from the details of the WHMCS 'Products/Services' tab. Otherwise, they will remain unchanged and set by default as in the template.
• DNS Domain - provide a DNS domain for the container. During VM creation, the setting from the host will be automatically used if neither a search domain nor a nameserver is set.
• Default User - provide the username that will be used for the VM instead of using the 'Service Username' field or the default user of the configured image. Otherwise, leave empty.
• Custom Cloud-Init Configuration - here you can optionally specify your custom config files that will be used instead of the automatically generated ones.
• Cloud-Init Script - you may also select one of your previously created in the addon scripts, it will be executed on VM creation or package change.
• Storage - the directory on which the CD-ROM ISO image is stored
• Template User - if enabled, the OS template first user will be used

Additionally, you can use the following commands that can be useful for debugging purposes:

• Re-run Cloud-Init scripts

rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/* &&
cloud-init init  &&
cloud-init modules -m final

• Logs

cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log

At the same time, you can monitor and manage each product from your WHMCS admin area.

1. Firstly, there are the standard module commands that you can perform on the VM.
2. Among them, you will find actions to:
   1. Manage the status of the machine,
   2. Access to remote consoles
   3. Migrate the server between nodes.
      You may enable 'Online' migration (no need to shut down the server before migration) and toggle 'Migration With Local Disks' which might take a while.
3. In this panel you will also find the basic information about the VM.
4. Moreover, you can change the VM hostname, ISO image, bootable devices as well as insert the SSH public key.

Note: The "Move Product/Service" action is not supported due to WHMCS limitations, and the fact that there are no hooks that would allow to catch such an event.

This option must be previously enabled in the product configuration, and successfully installed on the VM. Afterward, it should look similar to the image below.

Important: Create Network option works for KVM configuration only.

Note: The IP addresses you provide must already exist in the IP Management pool, and cannot be assigned to any other VM. This feature will not work if you already use the IP Manager For WHMCS module.

RAM details:

Assigned - the RAM assigned to the server (hosting services only)

Total - the full server RAM

Free - the difference between the Total and Assigned values

Suspended - the number of suspended accounts per size in MB E.g.: (1/500 MB) in case there is only one suspended account that takes up 500MB assigned to the Proxmox VPS server.

Press the 'More Details' icon, and you will obtain extended information on the selected server. Look below for more details.

To view information about the VM, simply press the "More Info" icon located on the right side of the page.

Important: Defined here node limits are strongly connected with the Load Balancer and node prioritization method used by the module to create new virtual machines.

• Allow To Create VM - if enabled, a VM can be created on this node
• Virtual Machines Number - define the maximum number of VM that can be created on this node
• Maximum Number of CPUs - set the top limit of CPUs on this single node
• Maximum Disk Size - set the top storage size allowed on this node
• Maximum RAM Size - set the maximum amount of RAM allowed on this node
• Default Storage - select the default storage for the VM created on this node
  

Setting node limits is crucial as creating virtual machines is largely based on them, once reached, a new virtual machine cannot be created on that node.

In order to convert a VM, press the 'Create Template' button.

Fill out the 'Description' field, and press 'confirm'.
Important: The selected VM will be destroyed in the process.

Note: VM ID range allows you to sort created machines depending on the server they were created on..
SSH Data - fill out the SSH connection details (Host, User, Posrt, Password or Private Key) to allow to locate the IP addresses on the Proxmox server (Cloud-Init)
Snippet Storage - select the catalog where the file will be uploaded (Cloud-Init)

Run the 'VM recovery synchronization' cron job to create and store full backups of VM configuration - you can find it in the point 26 of the configuration instructions.
Use the 'Export to File' button to save information in a TXT copy file.

Run the 'Tasks synchronization' cron job to synchronize the tasks, and view all tasks from the server - you can find it in the point 26 of the configuration instructions.

Important!: In order to create a virtual machine, you must previously add public IP addresses.
In case you do not own any public IP addresses in the database, you must previously enable the "cloud-init" option, only then it will be possible to create a VM with a private IP address.

• private/public address
• servers or nodes that will use them
• virtualization types (different VMs will use different IP pools)

Note: please go back to point 27 of the addon configuration to learn more details about adding IP addresses.

Keep in mind that only addresses that were automatically added when creating the VM. Removed IP addresses, that were the manually assigned, are not logged here.

By activating the IP Manager submodule, settings concerning IP addresses such as 'Configurable Options' and 'Default Options' will be ignored by Proxmox VPS.
Note: This integration works for LXC and KVM virtualization types.

1. Proceed to the IP Manager For WHMCS → Subnets tab in order to prepare the IP addresses subnet for Proxmox VPS For WHMCS.
2. Edit the previously created subnet, and create a custom field 'MAC Address' with a desired MAC address at each IP address in the subnet.
Note: Setting up the MAC address is optional and can be skipped.
More details about creating custom fields can be found here.

4. Now, link the configuration with Proxmox VPS product via a product, server or a configurable option.
On the following screen, we have carried out the linking via the configurable option assigned to the Proxmox VPS product, where each of the fields represents the number of assigned IP addresses.
You can, of course, mix them according to your needs, e.g. a single configurable option will assign the provided numbers for both IPv4 and IPv6 addresses.

Press 'Activate' next to the 'proxmoxVPS' module type as shown on the following screen.

Press 'Configure' , choose a configuration, and press 'Save Changes'.

Look above for the detailed instruction on how to proceed, follow the steps 1-6 to create a new custom field.
The only element you need to change is the custom field name, which should be 'Tag' in this case.

Set the weight of the resource that will prioritize its selection when sorting the nodes according to the free resources.
Note: The lowest possible weight is 1. You cannot set, e.g. 0.5.

To learn more about nodes and their allowed limits please go back to the Clusters section of this documentation.

Define the behavior of the tool:

• On Upgrade - select the action to perform:
  • None - no action will be triggered
  • Block - the upgrade process will be blocked as there are insufficient resources on the node
  • Migrate - the upgrade process will be performed after the previous migration of the VM to a server with sufficient free space
• Shutdown VM on Upgrade
• Stop VM If Shutdown Fails
• Enable to allow migrating the VM with local disks

Let's take exemplary weights for each resource just like the ones presented on the screen above.

VMs Number Weight: 1,5

CPU Weight: 2 

Disk Weight: 4

RAM Weight: 2,5 

"Disk Weight" with the highest weight, which means it has the highest priority, and "VMs Number Weight" as the least crucial factor on a given node.

In this case, there are four nodes created, named A, B, C, and D, with current resources as shown below:

Node A (Free Ram: 60%, Free CPU: 20%, Free Disk Space: 50%, Number of VMs: 5) 

Node B (Free Ram: 30%, Free CPU: 70%, Free Disk Space: 40%, Number of VMs: 10) 

Node C (Free Ram: 90%, Free CPU: 50%, Free Disk Space: 20%, Number of VMs: 20) 

Node D (Free Ram: 80%, Free CPU: 60%, Free Disk Space: 70%, Number of VMs: 15)

The next step is sorting the nodes according to each resource, it is done by assigning them corresponding priorities from 1 to 4.
The higher the priority, the more free resources available.

The process of sorting nodes is based on the following calculation method:

Final Node Priority = (RAM Priority x RAM Weight) + (CPU Priority x CPU Weight) + (Free Disk Priority x Disk Weight) + (No. VMs  Priority x VMs Weight)

The results for nodes A, B, C and D (with weights and sorting priority) are as follows:

Node A: (2 x 2,5) + ( 1 x 2 ) + ( 3 x 4 ) + ( 4 x 1,5 ) = 25
Node B: (1 x 2,5) + ( 4 x 2 ) + ( 2 x 4 ) + ( 3 x 1,5 ) = 23
Node C: (4 x 2,5) + ( 2 x 2 ) + ( 1 x 4 ) + ( 1 x 1,5 ) = 19,5
Node D: (3 x 2,5) + ( 3 x 2 ) + ( 4 x 4 ) + ( 2 x 1,5 ) = 32,5

As you can see, Node D has got the highest priority value with the result: 32,5.
It will be selected by Load Balancer to create a new VM on, or to migrate or upgrade an existing VM if its current node does not have sufficient resources.

The above described calculations are performed every time, 'Create' or 'Upgrade' actions are executed based on the current usage of resources on available nodes.

Important: In case a given client already has a VM existing on a node, this very node will NOT be taken into account during the prioritization of resources.
These security measures are taken to ensure that, in case of node failure, the client will still have access to the other VM server, created on another, properly working node.
At the same time, if this client has VMs on every node already, all of them will be taken into account during the prioritization process again.

If you decide to use the Load Balancer tool, define the resource weights to adjust the prioritization of nodes according to your needs.

• Proxmox VE Cloud VPS
  • Virtual Machines

• Proxmox VE VPS
  • Start
  • Stop
  • Reboot
  • Shutdown
  • Reinstall
  • Spice Console
  • noVNC console
  • Xterm.js Console
  • Information
  • IP Set
  • Graphs

• Both
  • Available Resources
  • IP Addresses
  • Migrate
  • User Details
  • Reconfigure Network

Note: The logs will be displayed only in case the errors occurred. If the task has finished successfully, there will be nothing to see.

As you can see, the control panel contains assorted options that will prove highly useful in VM management.
The module allows your customers to easily 'start', 'reboot', 'stop' and 'shutdown' their servers through pressing specific buttons.
It is also possible to 'reconfigure the network' without using the "Change Package" tool to easily update the network interfaces (KVM virtualization type only).
Clients can also access the 'noVNC' , 'SPICE' and 'Xterm.js' consoles. Among 'Additional Tools' , you will find options to 'reinstall' a server, create a 'backup' or set up 'backup jobs'.

Your clients are able to access resource usage 'graphs', view the VM 'history', manage the 'network' interface device, and set up the 'firewall' rules. Additionally, clients may take 'snapshots' , and manage 'disks'.
On the main product page in the client area, under the 'Information' headline, your clients can find such details as the current usage of CPU, memory, disk (LXC only) and SWAP (LXC only).
Customers can also view 'DNS' (LXC only) as well as assigned 'IP addresses'.

Note: The bellow example image was taken from the KVM-based VM, but it looks very similar in the case of LXC.

Important: If you do not see the Edit icon, it means the option to change hostname has been turned off by the admin.
Note: Information elements included in the below table are managed by the admin in the product's configuration section under Cleat Area Features.

Note: Since the 2.7.0 version of the module, your clients no longer need to download and install the client side SPICE application in order to connect with a VM. The console will be now opened in the virt-viewer instead.

You have to update the scripts on Proxmox v2.6.0. To do so, open the folder: UploadOnProxmoxServer/5.2-10 and upload the content on all your Proxmox servers.

You can allow your clients to either select a new OS from the provided KVM templates to automatically replace the system, or let them choose the ISO image for manual system installation.

Note: The ISO images can only be used for the KVM virtualization type, while VM templates can be used either for KVM or LXC.
Additionally, both KVM and LXC users can define the new OS password before a new installation.

Make sure you have set up the corn task as described in point 28 of the addon configuration instruction.

• Backup Size: used/allowed
• Backup Files: used/allowed
  

Next to the statistics there is a comfortable and easy to read progressive chart to illustrate the usage. If size is set to 'Unlimited', the chart does not show any progress.
Once you have reached the limits, the option to create new backups is blocked.

You can easily edit or delete, as well as preview the details of existing backup jobs there.
To add a new backup job, press 'New Job' as shown on the following screen.

Finally, press the 'Confirm' button.

They can also switch the timeline of graphs by choosing a new one from a dropdown menu indicated on the following screen. Clients can pick between 'Hour', 'Day', 'Week', 'Moth' and 'Year'.

If an error has occurred during the execution of a task, the client can view the API response error by clicking on the 'Information' icon.

Keep in mind that ' Network device' options have to be set to 'Private'.
Also, after the network creation, the button to add another one or delete a newly created private network may not be longer available, depending on your Proxmox configuration.

 php -q /your_whmcs/modules/addons/proxmoxAddon/cron/cron.php snapshots 

As you can see, you can easily modify and remove the previously added rules and security groups.
To add a new firewall rule press the 'Add' button.

The first way (orange) is to choose macro from the 'Macro' dropdown field.
The second way (blue) requires you to choose a protocol and fill in both the source and destination ports.
Additionally, you can enable a rule, and add a comment to it.
Press 'Add' to create a rule.

• Add Security group - choose a security group and an interface which the group will be used on.
  Additionally, you can enable a security group, and add a comment to it. Click 'Add' to use the security group.

• Download Rules - use this option to simply create a backup, once pressed, a file will be downloaded
• Restore Rules - restore any of the previously created backups created. Select one of the previously downloaded files with rules

• Name - enter the rule name
• Status - set it to enabled or disabled, can be changed later on
• Notifications Interval - define the time interval between notifications in hours. Eg. 8 - notifications will be sent in this frequency
• Data Timeframe - specify the time duration to extract and verify data usage (in minutes, eg. 30)
• Exceed All - if enabled, all the below setup conditions will have to be met to send the notification
• Email Address - if you want to send the notification to other addresses than the main customer's address, enter them here separated by a comma
• Conditions for:
  • Network Traffic [%] - sum of Net In and Net Out expressed in percentage value towards Rate Limit Network
  • CPU Usage [%] - a percentage that represents the overall utilization of CPU. It indicates how much of the CPU's processing capability is being utilized.
  • Memory Usage [%] - percentage of memory usage towards the VM RAM
  • Disk Read [%] and Disk Write [%] - disk usage percentage is calculated based on the Hard Disk Speed limits (respectively: Read Limit and Write Limit).
    If there are no limits set for Hard Disk Speed, the dedicated fields in the client area ('Disk Read [%]' and 'Disk Write [%]') will not be displayed

To begin configuring your first rule, simply click on "Add Task".

• Description - task description or short name
• Action - select the action the task is to execute:
  • Start the machine
  • Stop the machine
  • Reboot the machine
• Specify the exact time to keep the task active:
  • Start Date (use the calendar to specify the task beginning date)
  • Start Time - provide the exact hour to activate the task
  • End Date (use the calendar to specify the task ending date)
  • End Time - provide the exact hour to deactivate the task
• Job Type - choose the task type and define its execution frequency
  • One Time - the job will be executed only once, specify the time to start the task:
    • Job Hour
    • Job Minutes (rounded to 5 minutes)
  • Daily - the job will be executed every day of the week, specify the time to start the task:
    • Job Hour
    • Job Minutes (rounded to 5 minutes)
  • Weekday - the job will be executed on selected days, specify the time to start the task:
    • Job Hour
    • Job Minutes (rounded to 5 minutes)
    • Day of Week - select days when the task will be executed

Important: Storage Disc Space configurable option, when used by the client, overwrites the product configuration 'Disk Space ' option.

Go to 'Setup' → 'Products/Services' → 'Configurable Option' and press the edit button next to the configurable option used by your product.

Now, we will show you how to edit options. Let's say you want to add an option to order a VPS with 4 cores per socket.
To do so, edit 'The number of cores per socket' by pressing the edit button as shown on the screen below.

To add a new option, enter it in the textbox located below all other options. If you want to replace the option you currently have, simply edit the textbox.
Finish by pressing 'Save Changes'.

The table below lists all configurable options used by the module for the chosen virtualization type.
Note that the name on the left is a variable (which you should not edit), separated by the | from the editable custom name.

The module allows you to charge your customers based on the server resources they have used.
Note: Advanced Billing supports OpenVZ (prior to Proxmox v2.3.0), KVM and the LXC virtualization type.

You can easily use Proxmox VPS module with your existing Proxmox accounts. To do so, follow the instructions below:
1. Create and configure a Proxmox VPS server, and a server group.
2. Create and configure a Proxmox VPS product.
3. Place a new order with the previously created product for your customer. Do not activate the product yet.
4. Access the just created product from the admin area via the Client Profile → 'Products/Services' section and fill out 2 custom fields.
They are 'VMID' and 'Node' .

5. If possible, enter IP addresses into fields 'Dedicated IP' and 'Assigned IPs' .

6. Set the product status to 'Active' .
Remember not to click the 'Create' button during the entire process. That would unnecessarily create a new virtual machine.

In order to allow defining locations for Proxmox VPS, follow the below steps:
1. In your product's 'Module Settings' , set up 'Default Node' to 'Server-Node'.
2. Create a WHMCS server for each Proxmox node.
3. Manually create a configurable option, name it e.g.: 'Location'. Add options for each server within that configurable option.
4. Install our Server Allocator For WHMCS module and configure it properly.

It must assign a server to a product depending on the selected option in 'Location' configurable option.

Keep in mind that all locations need to have exactly the same configuration.

The "Create" action is triggered on the assigned server due to the 'Server-Node' option enabled.

When they are joined into one cluster, you can access all of them by connecting to just one.

• Auto-node - creates a new VM on the server (node) with the most free resources, so the administrator cannot control it
  
• Server-node - creates a VM on the server (node) whose IP address or node host (e.g. node1.yourproxmox.com) was provided during the WHMCS's server configuration,
  allowing the administrator to control the node choice

{$user.username}
{$user.password}
{$user.realm}

If you wish you may add other information, like password, into the email, just refer to e.g. 'VPS Server Welcome Email' email template in your WHMCS to find appropriate merge fields.
Important: Remember that if the client's email address is incorrect or WHMCS is not able to correctly send it because of any reason, the module will not create the VM for that client correctly.

Inside the admin area, in the product details view, you can preview the current IP Sets on VM:

• Service Creation Failed - if there were any errors that prevented the VM from being successfully created, then the selected email template will be sent to the administrator
• Upgrade - select the email notification template you want to receive if the client requests the product upgrade.
  This will also block the automatic sending of an upgrade request to the Proxmox API, so the administrator can review it first.
  Note that, if chosen, the product will still be upgraded in the WHMCS itself, but the upgrade request will not be passed to the API.
  To finalize the upgrade, use the "Change Package" module command in your admin area.
• To-Do List - if checked, in addition to sending any of the above notifications, it will also add them as tasks on the WHMCS To-Do List.

First, you need order Proxmox VPS product to your client account without processing the VM creation process.
Next, in the client's 'Products/Services' management tab of your admin area, fill the 'VMID' and 'Node' custom fields with the VMID of the virtual machine you want to import from your Proxmox server
Afterward, change the product status to 'Active' , save changes and use the 'Change User Role' module command.

Client: {$client_name} (ID: {$client_id}) 
Email: {$client_email} Service ID: {$service_id} 
Product: {$product_name} (ID: {$product_id})
Hostname: {$service_domain} 
Main IP: {$service_dedicated_ip} {if $service_assigned_ips} 
IP Address Allocation: {$service_assigned_ips} {/if}

 proxmoxHostname|Hostname 

• Create a VM template according to descriptions here
  
• Move to Module Settings → Virtual Machine Configuration → OS Template, select the previously created template (e.g. Ubuntu)
  
• In Module Settings → Cloud-Init: Cloud-Init: On, Service Username: On, Service Password: On
  

• Place a new order, wait until all tasks in Proxmox Addon → Tasks have 'finished' status.

• Create a new virtual machine using an ISO image in Proxmox panel
  
• Install Guest Agent on the windows system, read more here
  
• Convert the created machine into a KVM template
  
• Move to the product Module Settings → Guest Agent and switch options as below:

• Module Settings → Virtual Machine Configuration: select the previously created Windows template under OS Template
  
• Place a new order, wait until all tasks in Proxmox Addon → Tasks have 'finished' status.

You may assign a default user to a single template, that user will be automatically assigned to the newly created VM.

Important: To migrate the disc, you must previously 'Stop' your VM.
Note: Values provided in configurable option must be integers.

VLAN Tags Range Minimum and Maximum Numbers simply play the role of network separation measures. The minimum and maximum numbers cause the module to select the next free VLAN Tag from this range for each virtual machine it creates.
VLAN Tag is actually a drop-down list, and to select anything in this section, firstly, it is necessary to configure appropriate IP addresses in your Proxmox addon → IP Management section:

The module supports SDN private network type. It is possible to create virtual networks (vnets) at datacenter level thanks to the Software Defined Network (SDN).
To learn all the details on SDN in proxmox, please navigate to the official proxmox documentation.

In the module, the whole configuration of SDN is located in 'Module Settings' → 'Private Network' section. There you will find dedicated options, these are:

• Private Network Type
• SDN Zone
• SDN TAG Range Minimum Number
• SDN TAG Range Maximum Number

By default the parameter is set to 'true' and backups are loaded. You may change it manually to 'false' and no backups will be loaded in the 'Archive' dropdown list as on the screens below:

Follow the bellow pattern to add new lines:

$_LANG['template']['ISO_name'] = 'My ISO Name';  

For example:

$_LANG['template']['Ubuntu'] = 'Ubuntu - preferred';
$_LANG['template']['Debian Live 9.4.0 Amd64 Cinnamon'] = 'Aktuelles Debian';

Please note that ['ISO_name'] must be replaced with the full template or image name that you are going to change.
As a result, the OS templates and ISO images names will be displayed in the language of your choice or with your custom name.

 {$ipv4Addresses[0].gateway}

Please note that each time you update the module, you have to also update the files on the Proxmox server described in the step 4 of the installation guide.

Important: After upgrading the module to the newest version please first enter to the Proxmox addon in your WHMCS before setting up the product or opening it from admin or client area!
Otherwise, when you try to open the product's module settings, you may get SQLSTATE" "Base table or view not found" errors.

Please also note that since 2.7.0 module version the commands for the crons execution have slighty changed - although the old ones should still work, we recommend that you update their entries.

Keep in mind there is a quick and easy option in our client area that will let you upgrade the license of your module to an open source version at any time - and at a lower price!
To take advantage of it, simply use the License Upgrade button located on the product's page and a discounted invoice will be generated automatically.

Reason: The user account does not have permission to access the noVNC console.
Solution: Proceed to the product's 'Module Settings' tab and check whether the selected user's permission grants access to the noVNC console.
Afterward, proceed to the client's hosting and run 'Change User Role' on it.

To change the user role for all clients, please use the 'Cron job users synchronization (run only once)' cron command.
You will find it in the 'Proxmox Addon' → 'General Settings'.

Reason: Your certificate is not valid.
Quick Solution: Your client needs to add the Proxmox address to the certificate exception list.
Best Solution: Configure a valid certificate.

Reason: You have not uploaded the additional files on the Proxmox server.
Solution: Open 'UploadOnProxmoxServer' and upload content of the proper version on all Proxmox servers into the '/' path.

Reason: You have not created a client yet.
Solution: When the PAM authentication is used, it is necessary to create a client first in order to change the password correctly. Read more about it here.

Reason: You have not updated the php.ini file.
Solution: After SSH2 PECL installation, make sure that you have updated the php.ini files for CLI and webserver.

Solution: Turn off the Xterms.js console in the product configuration.

Reason: The user is other than "root" or the server authentication is set to Proxmox VE.
Solution: In the server configuration, set the root details, and select PAM as the authentication type.

Reason: An interface with the IP Address 10.10.10.10 cannot be found.
Solution: You need to provide your private IP Address node in the 'Assigned IP Addresses' field in the server configuration in your WHMCS.

Reason: Option 'Search For Templates On All Nodes' is enabled in the product configuration under KVM configuration.
Solution: Keep the above option disabled as shown on the screen.

Remember that cloning from node "A" to node "B" does not work on the local storage.
We do recommend using "Shared Storage" or creating the machine on the same node on which the KVM template is located.

It is important to keep every template named the same on every node, otherwise cloning will not work either, eg.:
node1: Ubuntu, CentOS, Debian
node2: Ubuntu, CentOS, Debian

Reason: The nameserver IP must be provided for the virtual machine of LXC configuration type.
By default, the module uses the content provided by the client during order, eg. ns1, then WHMCS completes the nameserver with a domain.
If the nameserver field is left empty or the client orders a new domain, the system is not able to process the data correctly and create a full address.
Solution: Enable the 'Use Server Nameservers' option in the product's configuration or manually correct the nameserver per the order.

Reason: The ProxmoxAddon_ServerUsage table has not been created properly because the tblservers table is on a different engine than "InnoDB"
Solution: Change the engine to "InnoDB", deactivate the proxmox addon and then activate it again. The missing table will be created.