Opnsense content filtering

[ejayb9]

Hello. I'm struggling to get content filtering working in Opnsense. I'm sure the part I have wrong is the rule order, in the NAT and in the interface Rules, but it could be something else.

Setup:
I'm using a 4-port Protectli and just updated Opnsense to 21.1.6.
Created a guest network on OPT2 with 192.168.1.1/24
No VLAN (I couldn't get this to work, and I couldn't find any tutorial on why it is even needed)
DHCP service for OPT2 is enabled.

Steps followed:
Added http://www.shallalist.de/Downloads/shallalist.tar.gz to Remote Access Lists - downloaded and apply
Cleared the authentication method and local group (within the Forward Proxy sub-menu)
Checked box to enable HTTP (Forward Proxy sub-menu). Turned on "full help" and the used the "add new firewall rule" link to add the NAT rule.
Select "add associated filter rule" at bottom of new NAT rule and also enabled rule.
Repeated for HTTPS.
Added HTTP and HTTPS block rules in OPT2
Checked box to enable proxy in Web Proxy -> administration -> general proxy settings
Done

Current Rule order for OPT2 is: 1)HTTP redirect to proxy rule 2)HTTPS redirect to proxy rule 3)HTTP block rule 4)HTTPS block rule
no other rules inside OPT2

It's just not working. Could use any help/advice. Thank you!

[Home Network Guy]

It sounds like you are trying to use the Web Proxy for a blocklist. I haven't tried using the Web Proxy for this purpose. I have created a few block lists but created a firewall alias which updates periodically and use a floating rule to block that list for all of my internal networks. The blocklist I use for this is a simple text file with IP addresses -- one on each line. (One list is the Spamhaus block list: https://www.spamhaus.org/drop/drop.txt)

When looking at that list you linked to, it is in a specific format that may only work for Squid and other types of firewall plugins/features that recognize that format. I don't know if the Web Proxy supports blocklists in that format.

[Home Network Guy]

I just took a quick look and that is one of the lists on the Web Proxy help page on OPNsense so that means it is in the right format. I apologize since I haven't looked into implementing the Web Proxy in my network so I don't have much experience with it. I would like to explore using it at some point, but I thought for my purposes, it may be helpful for transparently caching commonly downloaded files so it more than one device on my network needs the same files to download, it will be quicker to hit the local cache.

[ejayb9]

Thanks for the reply.
Yes, I used a combination of the Opnsense documentation with 2 Youtube videos
https://www.youtube.com/watch?v=EWGt6mWhN_o
https://www.youtube.com/watch?v=PmmzsKuEdCw

The first is for pfSense but combined with the other 2 sources it helps to fill in the blanks.

Wish there was more documentation and examples.

[Home Network Guy]

Yeah the documentation is lacking for several things. That is one thing that inspired me to create my site. Not just for OPNsense but for other more advanced home networking topics.

[ejayb9]

I wanted to reply to this in case anyone else tries to go down the route of content filtering and using a transparent proxy. It's basically useless. The reason, which I had to learn along the way, is because HTTPS cannot be decrypted, in order for the content to be filtered. I even tried to create a DNS based trust certificate on AWS and Let's Encrypt, but the certs are not updated fast enough to handle requests. So, I went with Sensei. As much as I hate to have to pay for a service, it at least makes things a bit easier to manager and configure. So, in that sense (pun intended), it's worth it.

[Home Network Guy]

Thanks for reporting back! I haven't tried to mess with web proxies especially since it seems like it works best for unencrypted traffic which isn't helpful most of the time since most everything is encrypted. I use Sensei also and it works well especially after they fixed a lot of the netmap issues that caused problems early on (at least with my hardware). I like being able to look at my traffic aggregated and broken down as well as block certain traffic.