Change Log

Foreword

You could follow this guide successfully to get up and running with an old desktop computer or a dedicated server. A few things might change on your end like disk partitioning or format types (eg. ZFS or ext4). Should mostly stay the same otherwise.

A Note on Virtualization

Type 1 Hypervisors

• These are called “bare metal” hypervisors, because there is nothing between the hypervisor and the computer’s hardware.
  • Installed directly on the disk
  • When you turn on the computer, the hypervisor is loaded
  • There is no other operating system
• Have 100% of the access to the computer’s hardware resources
• Common type 1 hypervisors:
  • VMware ESXi
  • Proxmox
  • XCP-NG

Type 2 Hypervisors

• Installed on top of an operating system like Windows, Linux, or Mac OS
• Do not have direct access to the computer’s hardware
• Cannot utilize 100% of the computer’s hardware resources, as they must be shared with the primary operating system
• When you turn on the computer, the primary operating system is loaded, then you run the hypervisor
• Common type 2 hypervisors:
  • VirtualBox
  • VMware Player
  • VMware Fusion

What We'll Build

Prerequisites

Host

• A dedicated host to run Proxmox VE
  • Laptop
  • Tower
  • Rackmount Server
• Recommended Hardware
  • RAM: 32 GB or more — runs out more quickly than you'd think
  • Storage: Multiple SSD Storage
    • Disk 1: 128 GB+ SSD
      • Proxmox VE installed here (smallest disk)
    • Additional Disk(s): 1 TB+ SSD
      • If more than one disk, can be configured into a striped zpool or configured with ZFS software RAID
      • If only one disk, no problem, as we'll walkthrough how to partition it for maximum potential during installation
  • CPU: 4+ cores (more is better for heavy compute workloads)

Network

• Recommended Hardware
  • Business-Grade Network Equipment
    • A business-grade router
    • A managed switch
    • Will allow you to create more sophisticated setups with 802.1q VLANs
  • Or, at the very least, an off-the-shelf router with more sophisticated configuration options
• This guide will walk through setting up remote access via VPN
  • At the moment, there are no instructions for working around CG-NAT

How can I check if I am behind CG-NAT?

• Login to your home router and find its public IP address
• Google search: what’s my IP
• Compare the two
• If they’re the same, good. If they’re different, you are behind CG-NAT

My Setup

Production Home Server

• Single Intel Xeon D-1541
• 128 GB RAM
• 256 GB SSD OS Disk
• 4 TB, 4 disk SATA array, ZFS Raid-Z2
• 4 port network interface card

The New Laptop Server

An HP EliteBook 840 G4

• Intel i7-7600U dual core, hyperthreaded = 4 vCPU
• 32 GB RAM (formerly 16 GB, was able to upgrade)
• 1 Intel gigabit Ethernet NIC
• 1 SSD, 500 GB

Installing Proxmox VE

Staging the Environment

Setting the Local Domain

I will be using my home router as an example. This procedure may vary from router to router. Most routers ship with a default private IP address range of 192.168.1.0/24. This can be changed at any time. I changed mine to be 172.16.1.0/24.

In my browser, I go to https://172.16.1.1 and login.

Then, I go to the Advanced > System Settings. I set my local domain to home.lab and save the changes.

Now, I can ping any device on my network using a local fully qualified domain name (FQDN)

# Pinging using the FQDN
ping lapprox.home.lab

Pinging lapprox.home.lab [172.16.1.16] with 32 bytes of data:
Reply from 172.16.1.16: bytes=32 time=2ms TTL=64
Reply from 172.16.1.16: bytes=32 time=1ms TTL=64
Reply from 172.16.1.16: bytes=32 time=1ms TTL=64
Reply from 172.16.1.16: bytes=32 time=2ms TTL=64


# We're on the same local network so using only the hostname works, too
ping lapprox

Pinging lapprox.home.lab [172.16.1.16] with 32 bytes of data:
Reply from 172.16.1.16: bytes=32 time=2ms TTL=64
Reply from 172.16.1.16: bytes=32 time=1ms TTL=64
Reply from 172.16.1.16: bytes=32 time=1ms TTL=64
Reply from 172.16.1.16: bytes=32 time=2ms TTL=64

Optional: Changing the Private IP Address Range

While still logged into the router, if you wish to change your private IP address range from the default to something else – like 172.16.1.0/24 – now is the time to do that.

Using my home router as an example (your procedure may vary).
Go to My Network > Network Connections > Network (Home/Office)
Go Settings and change the following values:

Don't forget to save the changes. The router may reboot after making these changes. If you were previously logged in at https://192.168.1.1, you will now be logged in at https://172.16.1.1.

You should do either of the following to facilitate that change:

A) Restart the client devices so they get a new IP address
B) Wait for the client DHCP lease to expire and they request a new address

Preparing the Laptop

Please remember that the steps I am taking here will probably be different from the steps you take to prepare your computer. This is just serving as a guide to get you in the right direction.

Creating a Bootable USB Drive

First, Download the Proxmox VE ISO.

Then, use a USB drive to make a bootable image using the .ISO file.

Enabling Virtualization in the BIOS

Turn on the laptop.
Press the ESC key multiple times until you get to the system setup menu.

Warning: you might need to press a different key on your system to get to the BIOS. For my HP laptop, ESC was the key to get to the BIOS.

Choose BIOS Setup

Go to Advanced > System Options

Enable VTx and VTd

Save changes and exit the BIOS setup

Boot from the USB Drive

Power off the laptop and insert the USB drive. Power on the laptop and press the ESC key until you reach the setup menu.

Choose the Boot Menu. Then, choose USB Disk.

Proxmox Installation Steps

Choose Install Proxmox VE (sorry, the font was very small). Down in the bottom-right, click I agree.

Disk Partitioning for Single Disk Installations

Select the install disk and click Options

I referenced the official documentation here: https://pve.proxmox.com/wiki/Installation regarding the partition sizes.

• hdsize: Defines the total hard disk size to be used.
  • In my case, I want to use the entire disk, so I specify 476.0
• swapsize: Defaults to the amount of RAM installed in the host
  • 12 GB is a reasonable choice, but the minimum is 4 GB
• maxroot: Defines the maximum size of the root volume, which stores the Proxmox VE operating system (Debian core files, logs, etc)
  • The maximum value is hdsize/4 — or in my case 476/4
  • However, I want to dedicate most of my space to maxvz, so I'll choose 75 GB
• minfree: Defines the amount of free space left in the LVM volume group pve
  • We can leave this empty in order to preserve space for maxvz
  • But, you could set it to something small like 16
• maxvz:
  • Make this value as large as you can!
  • This is where your VM and container storage will be placed
  • You calculate maxvz as such hdsize - maxroot - swapsize - minfree, so in my case 476 - 75 - 12 - 0 = 389

I will partition my 500 GB disk as such:

Click OK

Wrapping Up

Choose your country, time zone, and keyboard layout. Set a password for the root user. Set your email address – where you will receive alerts. Then, configure your network settings.

• Hostname: lapprox.home.lab (laptop Proxmox. Get it? Funny, right?)
• IP Address: I logged into my router and gave lapprox a static IP reservation of 172.16.1.16/24 and filled it out here.
• Gateway: Your home router's IP address
• DNS Server: Probably your home router, unless you want to specify otherwise

Confirm everything looks good and install. That's all there is to it.

Before We Begin

The Local Console

Proxmox VE is now installed on your laptop, desktop, server, mini PC, or whatever host you decided to use.

The Web Console

As seen above in the Local Console section, the message tells you to log into https://x.x.x.x:8006/ in your web browser.

In my case, I will be going to https://172.16.1.16:8006 on my personal computer You should see a login screen.

Some Required Changes

Open the shell from the web console. This is the same shell as if you are sitting in front of the server.

Update the APT Repositories

# Comment out the enterprise repositories
sed '/^[^#]/ s/^/# /' -i /etc/apt/sources.list.d/pve-enterprise.list
sed '/^[^#]/ s/^/# /' -i /etc/apt/sources.list.d/ceph.list
sed '/^[^#]/ s/^/# /' -i /etc/apt/sources.list.d/pve-enterprise.list.dpkg-dist
sed '/^[^#]/ s/^/# /' -i /etc/apt/sources.list.d/pve-enterprise.sources

# Add the community repositories
echo -e '\n# Proxmox community package repository' >> /etc/apt/sources.list
echo "deb http://download.proxmox.com/debian/pve $(grep CODENAME /etc/os-release | cut -d '=' -f 2) pve-no-subscription" >> /etc/apt/sources.list

Install Some Dependencies

apt clean && apt update
apt install ifupdown2 openvswitch-switch -y

If You're Using a Laptop

If you are following this guide, using an old laptop, you will want to disable power off when the lid is closed.

nano /etc/systemd/logind.conf

Change these lines from this:

#HandleLidSwitch=suspend
#HandleLidSwitchExternalPower=suspend
#HandleLidSwitchDocked=ignore

to this:

HandleLidSwitch=ignore
HandleLidSwitchExternalPower=ignore
HandleLidSwitchDocked=ignore

1. Press CTRL + X to exit
2. Press Y to save the changes
3. Press Enter to save with same file name

systemctl restart systemd-logind.service

Install Any Pending Upgrades

pveupgrade

Changing the Software Networking

Converting to Open vSwitch

By default, Proxmox uses Linux Bridges. There is nothing wrong with Linux Bridges. They just do not allow us to accomplish some of the things we could do with our home lab.

Make a Backup of the Current Network Configuration

cp /etc/network/interfaces /etc/network/interfaces.bak

Remove the Linux Bridge

Create the Open vSwitches

Create the Production Switch

Create the Management Interface

Create the pfSense Internal Switch

If for some reason you made a typo or some other error in your configuration and you’re having trouble connecting via the web browser:

1. Go to your Proxmox server and logon locally
2. Run these commands

cp /etc/network/interfaces.bak /etc/network/interfaces
ifreload -a

3. Try following these steps again

Trunk VLANs up VMBR1

Create the VLAN Zone

1. Click Datacenter
2. Click SDN
3. Click Zones

Create the VLAN Tags

Apply the SDN Settings

Tell Proxmox to Load the SDN Configurations

# Adds the line "source /etc/network/interfaces.d/*" to /etc/network/interfaces
# If grep does not find the line already exists
grep 'source /etc/network/interfaces.d/sdn' /etc/network/interfaces > /dev/null ||
grep 'source /etc/network/interfaces.d/\*' /etc/network/interfaces > /dev/null ||
echo -e '\nsource /etc/network/interfaces.d/*' >> /etc/network/interfaces

ifreload -a

Next Steps in the Series

Getting to Know Proxmox

Getting to Know Proxmox

In this module, we will take a look at some basic tasks and the core areas of the Proxmox web front end.

0xBEN0xBEN

Create a pfSense Firewall for Security Infrastructure

Create a pfSense Firewall for Our Proxmox Lab

In this module, we will look at setting up a pfSense firewall VM in Proxmox to segment our home lab network.

0xBEN0xBEN

Create a Kali Linux VM

Create a Kali Linux VM in Proxmox

In this module, we will look at the process of creating a Kali Linux VM using the command line in Proxmox

0xBEN0xBEN

Create an OWASP Juice Shop Container

Create an OWASP Juice Shop Container in Proxmox

In this module, we will look at creating a container in our Proxmox home lab to run OWASP Juice Shop to practice our web app pentest skills

0xBEN0xBEN

Adding Vulnhub VMs to the Cyber Range

Adding Vulnhub VMs to Our Proxmox Cyber Range

In this module, we will look at how to import VMs from Vulnhub into our Proxmox home lab

0xBEN0xBEN

Adding HackMyVM Boxes to the Cyber Range

Adding HackMyVm Boxes to Our Proxmox Cyber Range

In this module, we will look at how to import VMs from HackMyVM into our Proxmox home lab

0xBEN0xBEN

Setup Remote Access to Our Home Lab

Set up Remote Access to Our Proxmox Home Lab

In this module, we will look at using dynamic DNS, OpenVPN, and WireGuard to setup remote access to our Proxmox home lab

0xBEN0xBEN

Proxmox Troubleshooting

Troubleshooting Proxmox

In this module, we will look at various command line utilities to troubleshoot our Proxmox home lab

0xBEN0xBEN

Running Windows Guests on Proxmox

Running Windows Guests on Proxmox

In this module, we will look at the optimal way to run Windows guests in our Proxmox home lab

0xBEN0xBEN

Active Directory Lab: Getting Started

Adding an Active Directory Forest to Our Proxmox Lab

In this module, we will cover the steps to set up a small Active Directory forest in Proxmox, including a domain controller and two client computers

0xBEN0xBEN

Hack Your Active Directory Lab

Hack Your Proxmox AD Lab

In this module we will look at using a publicly available script to make our Active Directory domain susceptible to multiple vulnerabilities.

0xBEN0xBEN

Set up a Pivoting Lab

External Pentest Practice in Your Proxmox AD Lab

In this module of the Proxmox cybersecurity home lab project, we are going to look at the process of setting up a dual-homed target to serve as a pivot point into an Active Directory network.

0xBEN0xBEN

Adding a Comprehensive Wazuh SIEM and Network Intrusion Detection System (NIDS) to the Lab

Adding a Comprehensive Wazuh SIEM and Network Intrusion Detection System (NIDS) to the Proxmox Lab

In this module, we will take a look at the process setting up a comprehensive Wazuh SIEM, including a NIDS and some HIDS agents, in our Proxmox home lab.

0xBEN0xBEN

Creating a Windows 7 Buffer Overflow VM

Creating a Windows 7 Buffer Overflow Practice VM in Proxmox

In this module, we will look at creating a vulnerable Windows 7 VM in Proxmox that will run some applications which are vulnerable to 32-bit stack-based buffer overflows

0xBEN0xBEN

Keeping Tabs on Our Lab

Always-Running Resources

At any time, we are always going to want these resources running, so that our lab will function as intended. You should have these resources set to start at boot, just in case your computer loses power and comes back on.

On-Demand Resources

If we are running on a laptop with 32 GB RAM and 512 GB SSD storage, you can see that we have nearly maxed out the resources. It’s up to you how to proceed. Your options in this scenario would be:

• Increase the RAM on the server (if possible) and add another disk.
• Or, you could add another Proxmox node to your network and cluster them and then move the Active Directory lab to the other node.

We can keep tabs on our node’s resource consumption by glancing at the Summary panel, as well as the storage panels -- local and local-lvm.

To see how much space has been used in your local-lvm partition, you can scroll down to local-lvm (nodename) and then click Summary.

What Next?

Work within your constraints and look at ways you can improve what you already have.

Upgrade in Place

If you have a laptop with 16 GB of RAM and 1 TB disk space, I would consider the following:

• What is the maximum amount of RAM compatible with your system?
  • 32 GB or 64 GB?
    • Can you find a good deal on the memory?
      • Upgrade
    • If not, consider looking at another node.

Add Another Proxmox Node

That’s the nice thing about clustering multiple nodes together. If you have a couple of under-resourced nodes working together, you can accomplish more of your goals. You just have to budget out what works best for you.

Additional Lab Ideas

Other Proxmox Write-Ups

Proxmox - 0xBEN

Proxmox Virtual Environment and related services

0xBEN0xBEN

Malware Analysis
You could set up a malware analysis environment using some unpatched Windows VMs and the Remnux Linux distro.

Networking Labs
Set up some additional Linux routers and use IPTABLES or set up some additional pfSense firewalls and practice routing between them. You could set up some pivoting networks and practice SSH tunneling and data exfiltration.

Mail Server
Try hosting your own email system.

Web Server
Build a public or internal web server and try deploying a website to said server.

Clustering Proxmox Nodes

On the left, there is lapprox -- the laptop used in the making of this guide. On the right, there is my production Proxmox node. Lapprox is at 172.16.0.16 and my production server is at 172.16.0.6. I have joined them together to form a cluster. I can manage both servers from a single IP address. If I log into lapprox, I can manage both servers. If I log into production, I can manage both servers.

Visualizing the Cluster

Adding another PVE node and creating a cluster is a great way to add more CPU, RAM, and disk when low. You can then migrate existing VMs and LXCs to — and create new ones on — your new PVE node.

If you're still somewhat new to Proxmox, new to home labbing, or the concept of clustering servers is eluding you, I hope this diagram will help.

Preparing to Cluster

Home Firewall Before

• You probably had a consumer-grade firewall with an integrated Wi-Fi radio, likely issued by your Internet Service Provider
• A pfSense VM was sandwiched between vmbr0 and vmbr1
• The vtnet1 interace on the pfSense VM was the parent interface for 802.1q VLANs in the lab — such as vtnet1.80
• Any Proxmox VM or Linux Container (CT) plugged into vmbr0 received a DHCP IP configuration from the home router
• Any Proxmox VM or Linux Container (CT) plugged into vmbr1 received a DHCP IP configuration from the pfSense VM, depending on its assigned VLAN

Home Firewall After

• You've invested in a business-grade hardware firewall that supports 802.1q, perhaps pfSense
• The firewall doesn't have an integrated Wi-Fi radio, so you've also purchased a business-grade wireless access point that supports 802.1q-segmented SSIDs, perhaps Unifi
• The pfSense VM no longer exists
• The 802.1q VLANs have all been migrated to the hardware firewall, where a different interface is the parent as shown in the diagram — such as igb1.80
• All VMs and CTs are attached to vmbr0 and receive DHCP IP configurations from your new firewall depending on their assigned 802.1q VLAN ID

Home Switch Before

• You had a dumb switch, or plugged your PVE node into one of the integrated Ethernet ports on the consumer-grade firewall
• Any VM or CT attached to vmbr0 in Proxmox was effectively on your home network
• To segment certain VMs or CTs, you had to attach them to vmbr1 under the pfSense VM, and assign them specific VLAN IDs for the pfSense VM to manage
• Traffic from vmbr1 flowed up to pfSense, to vmbr0 if firewall rules allowed it, and to your dumb switch or router

Managed Switch After

• You've invested in a managed switch with 802.1q support
• Since the switch is managed, ports on the switch can be configured with 8021.q IDs, or left unconfigured as access ports for your Proxmox VMs and CTs
• All of your Proxmox nodes are plugged into this switch, and receive IP addresses from the DHCP server running on your new hardware firewall
• There is no more vmbr1, every VM and CT is attached to vmbr0 and receives access via your new managed switch
• All VLAN segmented traffic from the VMs and CTs flows up the trunk from vmbr0 to your new managed switch and then, to pfSense where firewall rules dictate the flow of traffic
• All VLAN segmented traffic from the wireless access point flows up the trunk to the managed switch and then, to pfSense where firewall rules dictate the flow of traffic

The Clustering Process

I would highly advise reading this page before proceeding: https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_join_node_to_cluster

1. Go to your existing node (eg. https://existing-node-ip:8006)
   This is your Proxmox node that already has guests and configurations.
   • Click on Datacenter > Cluster
   • Click on Create Cluster
   • Click on Join Information
   • Click Copy Information
2. Go to your brand new node (eg. https://172.16.0.16:8006)
   This is your brand-new Proxmox node. It is empty, no guests, no configurations.
   • Go to Datacenter > Cluster
   • Click Join Cluster
   • Paste the Join Information you copied above
   • And, join the cluster

Other Projects

Proxmox Lab: Game of Active Directory

In this project, we’ll be using deployment tools and scripts shared by m4yfly to standup and configure Game of Active Directory (GOAD) in the environment set up in my original Proxmox lab series.

0xBEN0xBEN

Building a Security Lab in VMware Workstation Pro

In this project, broken up into multiple modules, you will build a comprehensive cybersecurity home lab using VMware Workstation Pro. Upon completion, you will have an environment where you can safely practice penetration testing against a wide variety of targets, as well as detection in your SIEM.

0xBEN0xBEN

Building a Security Lab in VirtualBox

In this project, we we will take a look at an in-detail process of setting up an entry-level cybersecurity lab using VirtualBox

0xBEN0xBEN

Honorable Mentions

• @Tank1334 for QA testing and feedback
• @dfir-jesseee from the Black Hills Information Security Discord server for QA testing and feedback
• @Jj19101 for QA testing and feedback
• @Blund from the TCM Security Discord Server for QA testing and feedback
• @Admiral Awesome from the InfoSec Prep Discord Server for QA testing and feedback
• @ip3c4c for QA testing, feedback, and his own additions to this lab guide documented at his blog
• @Khiz3r for QA testing and feedback