Archive for the ‘freebsd’ Category

Managing ports for multiple FreeBSD servers

Monday, July 31st, 2017

This is a follow up post on how to manage ports for multiple FreeBSD servers. If you’re looking for how to update the operating system itself, have a look at my almost three years old post: Managing multiple FreeBSD servers.

Alright, so what we’re trying to solve is this: multiple VMs running the same (or different) release of FreeBSD, and you’re looking for a way to centralize delivery of packages to your FreeBSD VMs.

(more…)

make buildworld & IBM x3550 m3

Saturday, June 24th, 2017

Upgrade from 11.0-RELEASE to 11.1-BETA3

Brand: IBM x3550 m3

In order to successfully boot the server make sure to enable legacy support as per this thread.

Processor: 2 x Intel Xeon E5620 2.40GHz (4 cores each)
Memory: 8GB
HDD: 2 x 146GB (10k RPM, 6Gbps SAS 2.5-inch) in RAID1

Softupdates: ON
SMP: ON

  1. CPU: Intel(R) Xeon(R) CPU E5620  @ 2.40GHz (2400.13-MHz K8-class CPU)
  2. real memory  = 8589934592 (8192 MB)
  3. avail memory = 8244543488 (7862 MB)
  4. mfi0: <LSI MegaSAS Gen2> port 0x1000-0x10ff mem 0x97940000-0x97943fff,0x97900000-0x9793ffff irq 16 at device 0.0 on pci11
  5. mfi0: Using MSI
  6. mfi0: Megaraid SAS driver Ver 4.23
  7. mfi0: FW MaxCmds = 1008, limiting to 128
  8. mfid0: 139236MB (285155328 sectors) RAID volume (no label) is optimal

make -j4 buildworld: 1h 36m 28s
make -j4 buildkernel: 5m 58s
make installkernel: 13s
make installworld: 3m 32s

MySQL cluster using ndbcluster engine under FreeBSD 11

Sunday, January 22nd, 2017

!SPOILER ALERT!

Not to discourage you, but make sure you read and understand MySQL cluster limitations thoroughly prior to start building the cluster. You don’t want to spend time on building the whole thing just to discover at the very end that you hit some hard coded limitation that can’t be resolved. It’s very easy to be trapped into Catch-22 here: some third-party vendor might say “why would we want to adjust our software to overcome MySQL limitations?”, and I’m sure MySQL dev team has had valid reasons to introduce those. So you end up in the middle, and you’re basically stuck.

For example, the vanilla typo3 distribution won’t work with ndbcluster engine out of the box. You hit the Row size limitation almost immediately, and unless you’re willing to spend time to analyze and optimize the structure of the typo3 database you’re blocked. You might be lucky, and it could be just a small change from varchar(2000) to varchar(1000), but you might be not. In addition to that, you’ll most certainly need a separate instance of MySQL with InnoDB or MyISAM, so you can import the DB, dump it, and start feeding it to the ndbcluster engine in batches. All these contribute to the time spent, and during the course of the installation you start considering alternatives, like changing the Operating System, and/or trying Galera for instance, or even switching to PostgreSQL altogether, but we’re not looking for easy paths, are we? :)

(more…)

NSD and OpenDNSSEC under FreeBSD 10 [Part 5: SafeNet HSM]

Wednesday, May 18th, 2016

This is the fifth part in the series of articles explaining how to run NSD and OpenDNSSEC under FreeBSD 10.

This time we’re going to integrate proper hardware HSM support in our setup — a pair of SafeNet Network HSMs (aka Luna SA).

Here is how our updated installation diagram looks like:

2016051701

Before we jump into technical details there are a couple of assumptions:

— I assume that HSMs are already configured and partitioned. HSM installation is outside of scope of this guide since it’s a lengthy and pretty time consuming process which has nothing to do with OpenDNSSEC. It also involves a big chunk of work to be done on the access federation field (different teams accessing different partitions with different PEDs or passwords). SafeNet HSM’s documentation is quite solid though, so make sure this part is completed. In our setup, both HSMs run the latest software 6.2.0-15 and there is one partition created on both units called TEST. TEST partition is activated and we’re going to create High Availability group, add both HSMs to the HA group and allow NS-SIGN to access it;

— As you might have noticed, I decided to leave ZSKs to be handled by SoftHSM. One of the things that you’ll have to keep an eye on with network HSMs is the HDD space. The way it works with SafeNet is that you have an appliance with some fixed amount of disk space (let’s say 2MB). Then you create partitions and allocate space out of total amount for each partition (by default it’s equal distribution). So let’s assume we created five partitions 417274 bytes each. Normally, storing a pair of public/private key consumes very little, but with OpenDNSSEC we’re talking about a number of domains each storing a pair of public/private keys for both KSK and ZSK. It’s very important to understand how far you can go, so you’re not surprised after several years when you discover that you run out of space.

Let’s do some basic math: one domain, with both ZSK (1024) and KSK (2048) stored on HSM, will consume 2768 bytes, so with 417274 bytes partition you should be able to handle ~150 domains. However, during ZSK or KSK rollover, another pair will be temporarily created, and although ZSK/KSK rollover shouldn’t happen at the same time and OpenDNSSEC will purge expired keys after the rollover is completed, you’ll have to consider extra 2768 bytes per domain (for a period of time defined in <Purge> stanza in kasp.xml), which leaves you 75 domains. As you can see this isn’t much. That’s why I decided to keep SoftHSM for ZSKs to save some HSM space (which is not cheap to say the least!).

One of the disadvantages of keeping both storage engines is that you’ll have one more dependency to worry about should you consider to upgrade (for example to SoftHSM2), hence the choice is yours. Another option would be to store private keys in HSM and leave public keys aside (<SkipPublicKey/> option under conf.xml), but I’ve read that it’s very much dependent on the HSM provider and could lead to unexpected results. And one more option would be to use <ShareKeys/> under kasp.xml — that way you can share the same key for multiple domains.

(more…)

FreeBSD template for ManageEngine OpManager

Friday, March 18th, 2016

We use OpManager by ManageEngine to monitor our infrastructure. Most of Linux flavors are already covered by default templates in OpManager. Moreover, you’ll be able to get interface statistics and CPU/RAM utilization of FreeBSD servers with the included UCD SNMP MIBs. The only bit that was missing was the monitoring of partitions for FreeBSD, hence I decided to spend a bit of my time and finally make the template that could be used in OpManager to monitor FreeBSD servers.

It’s confirmed to work with the latest OpManager 11 (build 11600) and FreeBSD 10.x without UCD Net-SNMP installed but only bsnmp with bsnmp-ucd. The reason why bsnmp is simple: bsnmp is light and is part of the base FreeBSD, so you don’t need to install anything and bsnmp-ucd (available under /usr/ports/net-mgmt/bsnmp-ucd) is a module for bsnmpd which implements parts of UCD-SNMP-MIB, while UCD Net-SNMP requires a massive amount of dependencies to be installed.

Once bsnmp-ucd is installed you might want to enable ucd module in /etc/snmpd.config and restart bsnmpd:

  1. # UCD module
  2. begemotSnmpdModulePath."ucd" = "/usr/local/lib/snmp_ucd.so"

So here we go (you can also download it from here, just make sure to change the extension to XML):

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <CustomDevicePackage>
  3. <CustomDevicePackage deviceType="FreeBSD" iconName="linux.png" pingInterval="5">
  4. <SysOIDs>
  5. <SysOID oid=".1.3.6.1.4.1.12325.1.1.2.1.1"/>
  6. </SysOIDs>
  7. <GRAPHDETAILS>
  8. <Graph SaveAbsolutes="false" YAXISTEXT="Percentage" customGraph="false" description="Monitors the Memory utilization based on UCD SNMP MIB" displayName="Memory Utilization(UCD SNM MIB)" failureThreshold="1" graphID="966" graphName="Lin-MemoryUtilization" graphType="node" isNumeric="true" oid="(.1.3.6.1.4.1.2021.4.5.0-.1.3.6.1.4.1.2021.4.6.0-.1.3.6.1.4.1.2021.4.14.0-.1.3.6.1.4.1.2021.4.15.0)*100/.1.3.6.1.4.1.2021.4.5.0" period="900" protocol="SNMP" sSave="true" timeAvg="false">
  9. <OTHEROIDS/>
  10. </Graph>
  11. <Graph SaveAbsolutes="false" YAXISTEXT="Percentage" customGraph="false" description="Monitors the CPU Utilization based on UCD SNMP MIB" displayName="CPU Utilization(UCD SNMP MIB)" failureThreshold="1" graphID="315" graphName="Lin-CPUUtilization" graphType="node" isNumeric="true" oid=".1.3.6.1.4.1.2021.11.9.0" period="900" protocol="SNMP" sSave="true" timeAvg="false">
  12. <OTHEROIDS/>
  13. </Graph>
  14. <Graph DisplayColumn=".1.3.6.1.4.1.2021.9.1.2" Index=".1.3.6.1.4.1.2021.9.1.1" SaveAbsolutes="false" YAXISTEXT="Percentage" customGraph="false" description="Monitoring the usage in each partition of the FreeBSD Device." displayName="Partition Details of the FreeBSD Device (%)" failureThreshold="1" graphID="252000" graphName="BSDPartitionWiseDiskDetails" graphType="multiplenode" isNumeric="true" oid="(.1.3.6.1.4.1.2021.9.1.8*100/.1.3.6.1.4.1.2021.9.1.7)" period="900" protocol="SNMP" sSave="true" timeAvg="false">
  15. <OTHEROIDS/>
  16. </Graph>
  17. </GRAPHDETAILS>
  18. <Category name="Server"/>
  19. <Vendor name="net-snmp"/>
  20. <Version version="2016031804"/>
  21. </CustomDevicePackage>
  22. </CustomDevicePackage>

Noteworthy sections:

SysOID oid=: this is the FreeBSD system identifier. When you’re going to add a new FreeBSD server the template will be automatically attached based on SysOID.

CPU and RAM sections were copied from the standard Linux template.

DisplayColumn=: .1.3.6.1.4.1.2021.9.1.2 is a list of available partitions (/, /usr, /var, etc.).

Index=: .1.3.6.1.4.1.2021.9.1.1 is a list of IDs of available partitions.

oid=: (.1.3.6.1.4.1.2021.9.1.8*100/.1.3.6.1.4.1.2021.9.1.7) is used to calculate the percentage of utilization of a particular partition, where .1.3.6.1.4.1.2021.9.1.8 is used space and .1.3.6.1.4.1.2021.9.1.7 is available space.

Hope it helps.

Managing multiple FreeBSD servers

Monday, November 17th, 2014

If you run multiple installations of FreeBSD sooner or later you will face with the issue of how to update them all in the most efficient and centralized way. Building kernel/world for a FreeBSD server with one CPU and couple of GB of RAM will take hours to complete. Fortunately, there is a way to optimize it.

(more…)

User based access control for Skype with Dante and FreeBSD 10

Saturday, October 4th, 2014

[20170903/ Yet another one. Since I rarely (read never) use Skype to call, I couldn’t spot it immediately, but all calls were actually dropped after 10 seconds. To fix it, you’ll need to open 3478/udp from your LAN to the outside world. More details are here. Confirmed to work with 7.39.32.102./20170903]

[20170512/ Another update. It actually came out that the latest Skype (confirmed with 7.36.0.101) does support socks, however there is an issue with the password length used to authenticate to the socks instance. Anything greater than 5 (five) characters fails. I don’t know whether it’s done deliberately, or there is some bug that’s never going to be fixed, but anyway, the workaround is either to use some generic account with the five-or-less-characters password, or disable socks authentication altogether./20170512]

[20170312/ Somewhere around first week of March 2017 (or end of February), Skype started dropping connections from versions 7.1.32.xx and below. When you try to log in, you’ll be presented with the message about outdated version. I’m not aware about any versions after 7.1.32 that support Socks. Despite of numerous bugs opened Socks functionality was never fixed, therefore the content of this article is no longer valid, and you won’t be able to use Skype with Socks. The configuration option is still there, but no connection attempts to the Socks server are made. Perhaps this is how they promote the usage of Skype Business Server./20170312]

Today we’re going to configure Dante running on FreeBSD 10.0-STABLE to allow Skype connectivity based on username/password stored in Active Directory. The version of Dante being used is 1.4.1 installed from ports and Active Directory is handled by Windows Server 2008 R2.

Note: as at the time of writing, the version of Dante available in FreeBSD ports collection is 1.4.0 and it’s marked as BROKEN because of the bug 192295. Use this patch to install 1.4.1.

Install required software:

  1. % cd /usr/ports/security/pam_ldap && make install clean
  2. % cd /usr/ports/net/dante && make install clean

Create /usr/local/etc/pam.d/sockd file:

  1. auth       required /usr/local/lib/pam_ldap.so
  2. account    required /usr/local/lib/pam_ldap.so
  3. password   required /usr/local/lib/pam_ldap.so

Create /usr/local/etc/ldap.conf file and fix permissions:

  1. host 10.9.128.1
  2. base OU=Users,DC=int,DC=domain,DC=org
  3. ldap_version 3
  4. binddn CN=socksd,OU=Users,DC=int,DC=domain,DC=org
  5. bindpw xxxxxxx
  6. pam_filter objectclass=user
  7. pam_login_attribute samaccountname
  1. % chmod 600 /usr/local/etc/ldap.conf

Adjust host, base, binddn and bindpw to reflect your environment.

Modify /usr/local/etc/sockd.conf file:

  1. logoutput: stdout /var/log/dante.log
  2. internal: 10.9.36.10 port = 1080
  3. external: 10.9.36.10
  4.  
  5. socksmethod: pam.username none
  6.  
  7. user.privileged: root
  8. user.unprivileged: nobody
  9. user.libwrap: nobody
  10.  
  11. client pass {
  12.         from: 10.9.128.0/24 port 1024-65535 to: 0.0.0.0/0
  13.         log: error connect disconnect
  14. }
  15.  
  16. socks pass {
  17.         from: 10.9.128.0/24 to: 0.0.0.0/0
  18.         command: connect udpassociate
  19.         socksmethod: pam.username
  20.         log: error connect disconnect iooperation
  21. }
  22.  
  23. socks pass {
  24.         from: 0.0.0.0/0 to: 10.9.128.0/24
  25.         command: udpreply
  26.         log: connect error
  27. }

Modify /etc/rc.conf to start Dante at boot:

  1. # enable dante
  2. sockd_enable="YES"

Configure Skype to use Socks with proxy authentication and check the logs of Dante:

  1. Oct  4 16:45:44 (1412433944.407661) sockd[482]: info: pass(1): tcp/accept [: 10.9.128.47.59574 10.9.36.10.1080
  2. Oct  4 16:45:44 (1412433944.525454) sockd[483]: info: pass(1): udp/udpassociate [: pam.username%usera@0.0.0.0.64526 10.9.36.10.60780
  3. Oct  4 16:45:49 (1412433949.196857) sockd[630]: info: pass(1): tcp/accept [: 10.9.128.47.59578 10.9.36.10.1080
  4. Oct  4 16:45:49 (1412433949.378804) sockd[499]: info: pass(1): tcp/connect [: pam.username%usera@10.9.128.47.59578 10.9.36.10.1080 -> 10.9.36.10.59578 64.4.23.143.33033
  5. Oct  4 16:45:49 (1412433949.380925) sockd[499]: info: pass(1): tcp/connect -: pam.username%usera@10.9.128.47.59578 10.9.36.10.1080 -> 10.9.36.10.59578 64.4.23.143.33033 (35)
  6. Oct  4 16:45:49 (1412433949.544730) sockd[499]: info: pass(1): tcp/connect -: 64.4.23.143.33033 10.9.36.10.59578 -> 10.9.36.10.1080 pam.username%usera@10.9.128.47.59578 (63)
  7. Oct  4 16:45:49 (1412433949.548340) sockd[499]: info: pass(1): tcp/connect -: pam.username%usera@10.9.128.47.59578 10.9.36.10.1080 -> 10.9.36.10.59578 64.4.23.143.33033 (40)
  8. Oct  4 16:45:49 (1412433949.706747) sockd[499]: info: pass(1): tcp/connect -: 64.4.23.143.33033 10.9.36.10.59578 -> 10.9.36.10.1080 pam.username%usera@10.9.128.47.59578 (3)
  9. Oct  4 16:45:49 (1412433949.962844) sockd[499]: info: pass(1): tcp/connect -: 64.4.23.143.33033 10.9.36.10.59578 -> 10.9.36.10.1080 pam.username%usera@10.9.128.47.59578 (144)

Since Dante is quite talkative make sure to rotate logs by editing /etc/newsyslog.conf file:

  1. /var/log/dante.log                      640  3     100  *     JB    /var/run/sockd.pid

Running Splunk with add-on for Check Point OPSEC LEA on FreeBSD 10

Tuesday, September 30th, 2014

Here is my set of notes to run latest Splunk with add-on for Check Point OPSEC LEA on FreeBSD 10.0-STABLE.

Splunk: 6.1.3 build 220630
Add-on for Check Point OPSEC LEA: 2.1.0
Check Point: R71.30

(more…)

make buildworld & IBM x3650 m3

Sunday, September 28th, 2014

Upgrade from 10.0-RELEASE to 10.1-PRERELEASE

Brand: IBM x3650 m3

In order to successfully boot the server make sure to enable legacy support as per this thread.

Processor: 2 x Intel Xeon X5650 2.67Ghz (6 cores each)
Memory: 144GB
HDD: 2 x 72GB (15k RPM, 6Gbps SAS 2.5-inch) in RAID1

Softupdates: ON
SMP: ON

  1. CPU: Intel(R) Xeon(R) CPU X5650  @ 2.67GHz (2666.82-MHz K8-class CPU)
  2. real memory = 154618822656 (147456 MB)
  3. avail memory = 150235295744 (143275 MB)
  4. mfi0: <LSI MegaSAS Gen2> port 0x1000-0x10ff mem 0x97940000-0x97943fff,0x97900000-0x9793ffff irq 16 at device 0.0 on pci1
  5. mfid0: 68664MB (140623872 sectors) RAID volume (no label) is optimal

make -j4 buildworld: 31mm 20ss
make -j4 buildkernel: 03mm 31ss
make installkernel: 09ss
make installworld: 58ss

NSD and OpenDNSSEC under FreeBSD 10 [Part 4: Firewall]

Sunday, August 31st, 2014

This is the forth part in the series of articles explaining how to run NSD and OpenDNSSEC under FreeBSD 10.

It’s just a reference for firewall freaks like me showing which ports need to be opened on each respective server.

I’m using PF on all four servers with the ‘deny all’ default rule.

NS-FEED needs to be able to send 53/udp to the signer and both slaves (to NOTIFY about zone changes). It has to accept incoming 53/tcp from the signer and both slaves (to allow zone transfers) and also it has to accept 53/udp from the signer (the signer will periodically poll the hidden master for zone changes):

  1. pass out quick on xl0 inet proto udp from xl0 to 192.168.128.1 port = domain keep state
  2. pass out quick on xl0 inet proto udp from xl0 to 192.168.128.2 port = domain keep state
  3. pass out quick on xl0 inet proto udp from xl0 to 10.9.128.2 port = domain keep state
  4.  
  5. pass in quick on xl0 inet proto tcp from 192.168.128.1 to xl0 port = domain flags S/SA keep state
  6. pass in quick on xl0 inet proto tcp from 192.168.128.2 to xl0 port = domain flags S/SA keep state
  7. pass in quick on xl0 inet proto tcp from 10.9.128.2 to xl0 port = domain flags S/SA keep state
  8.  
  9. pass in quick on xl0 inet proto udp from 10.9.128.2 to xl0 port = domain keep state

NS-SIGN has to be able to send 53/udp and 53/tcp to the hidden master (to request and perform zone transfer from the hidden master). It has to be able to send 53/udp to both slaves (to notify slaves about zone changes). It has to accept incoming 53/tcp from both slaves (to allow zone transfers) and also it has to accept 53/udp from the hidden master:

  1. pass out quick on xl0 inet proto tcp from xl0 to 10.9.128.1 port = domain flags S/SA modulate state
  2. pass out quick on xl0 inet proto udp from xl0 to 10.9.128.1 port = domain keep state
  3. pass out quick on xl0 inet proto udp from xl0 to 192.168.128.1 port = domain keep state
  4. pass out quick on xl0 inet proto udp from xl0 to 192.168.128.2 port = domain keep state
  5.  
  6. pass in quick on xl0 inet proto tcp from 192.168.128.1 to xl0 port = domain flags S/SA keep state
  7. pass in quick on xl0 inet proto tcp from 192.168.128.2 to xl0 port = domain flags S/SA keep state
  8. pass in quick on xl0 inet proto tcp from 10.9.128.1 to xl0 port = domain flags S/SA keep state

NS-3 and NS-4 have to be able to send 53/tcp to the hidden master and the signer (to request zone transfers). They also have to accept incoming 53/udp and 53/tcp from anywhere to serve recursive DNS requests:

  1. pass out quick on xl0 inet proto tcp from xl0 to 10.9.128.1 port = domain flags S/SA modulate state
  2. pass out quick on xl0 inet proto tcp from xl0 to 10.9.128.2 port = domain flags S/SA modulate state
  3.  
  4. pass in quick on xl0 inet proto udp from any to 192.168.128.1 port = domain keep state
  5. pass in quick on xl0 inet proto tcp from any to 192.168.128.1 port = domain flags S/SA modulate state
  1. pass out quick on xl0 inet proto tcp from xl0 to 10.9.128.1 port = domain flags S/SA modulate state
  2. pass out quick on xl0 inet proto tcp from xl0 to 10.9.128.2 port = domain flags S/SA modulate state
  3.  
  4. pass in quick on xl0 inet proto udp from any to 192.168.128.2 port = domain keep state
  5. pass in quick on xl0 inet proto tcp from any to 192.168.128.2 port = domain flags S/SA modulate state