Ntopng установка на pfsense
Содержание:
- nProbe Cento
- Installation of ntopng
- nDPI
- Statistics¶
- Sites¶
- User Activities¶
- PF_RING (5)
- Applications¶
- Custom Applications¶
- Restore Configuration¶
- TLS¶
- nProbe (4)
- Payments (2)
- ntop (5)
- Licensing¶
- nBox
- Custom Categories¶
- Running Multiple Daemons¶
- About Us
- Automatic Daemon Startup on Boot¶
- Promiscuous Interfaces
- Using Behind a Firewall¶
- Home¶
nProbe Cento
A 1/10/40/100 Gbit NetFlow/IPFIX Probe, Traffic Classifier, and Packet Shunter
nProbe Cento is a high-speed NetFlow probe able to keep up with 1/10/100 Gbit. nProbe Cento is not just a fast Netflow probe, it has been designed as the first component of a modular monitoring system: besides capturing ingress packets and computing flow data, it can be used to classify the traffic via DPI (Deep Packet Inspection) and performs optional actions on selected packets/flows when used as traffic forwarder in combination with other applications such as IPS/IDS, traffic recorders, etc.
Features
- NetFlow v5/v9/IPFIX support
- Flow export to JSON, Text, Kafka, Syslog, ntopng
- IPv4 and IPv6 support
- Native PF_RING and PF_RING ZC support for high-speed packet processing
- Support of Accolade, Intel, Myricom, Napatech 10/40/100 Gbit NICs, and Silicom/Fiberblaze 10/100 Gbit NICs
- Scalable, multithreaded design, ingress traffic can be load balanced across multiple streams on multi-core architectures
- Layer-7 application visibility using nDPI (Deep Packet Inspection) or micro-nDPI (a lightweight DPI library supporting the most important protocols such as HTTP/HTTPS/DNS) for improved performance
- Flow-based Load Balancing to IDS/IPS (Snort, Bro, Suricata)
- Traffic filtering based on protocols to reduce the load on the IDS
- Feedback channel for traffic filtering/shunting from an IPS: “forward this flow”, “drop this flow”, “divert this flow through the IPS”
- Traffic filtering and slicing for saving storage space removing meaningless data when forwarding traffic to a packet-to-disk application such as n2disk
Use Cases
nProbe Cento can be used in multiple configurations. Following is a non-exhaustive list of possible use cases.
Probe and Inline Traffic Bridge with Layer-7 Filtering
Performance
nProbe Cento has been designed to keep up with 1/10/100 Gigabit speeds on commodity hardware. On adequate hardware nProbe Cento is able to process 10 Gbit per physical core, and to scale almost linearly in the number of cores (as long as there is enough memory bandwidth available) as shown in the picture below.
In order to process 100 Gbit it is possible to load-balance the traffic across multiple cores using RSS-like techniques usually available on Intel cards or specialised packet-capture cards.
The picture below shows the result of a performance test using a 100 Gbit Napatech card configured with 26 strams. The total processed rate (sustained) was of 132 Mpps with 70-byte packets (0 drops). The configuration used for the test:
- nProbe Cento (native PF_RING support)
- CentoOS 6.x x64
- PF_RING 6.3.X
- 2 x CPU Intel E5 v3
- Napatech 100 Gbit card
- 500K rotating IP addresses
- Generation of 25 million flows/minute
- No flow storage on DB or disk, just forwarding (in NFv9 format) to a collector
For the latest news about nProbe Cento, please read the ntop blog.
License
nProbe Cento is distributed under the EULA and requires a license per system. Licenses are available in various flavours depending on the number and speed of network interfaces Cento can handle per system.
| Max Concurrent Ports | Cento S | Cento M | Cento L | Cento XL | Cento XXL |
|---|---|---|---|---|---|
| 1 Gbit | 2 | Unlimited | Unlimited | Unlimited | Unlimited |
| 10 Gbit | 2 | Unlimited | Unlimited | Unlimited | |
| 40 Gbit | 2 | Unlimited | Unlimited | ||
| 100 Gbit | 2 | Unlimited |
nProbe Cento is available in our repositories. You can purchase online your copy of nProbe Cento at the ntop e-shop.
Installation of ntopng
I am using a fresh Ubuntu Server 14.04 LTS edition (64-bit <- which is required for ntopng). As always I am installing a few basic software packages before starting with the actual service. The packages for ntopng can be found here. Select either the “nightly” or “stable” builds. For more reliable versions, you should choose the stable one. Execute the following two commands on the server to add the repository of ntopng:
|
1 |
wget httpapt-stable.ntop.org14.04allapt-ntop-stable.deb sudo dpkg-iapt-ntop-stable.deb |
Have a look at “/etc/apt/sources.list.d/”. There is now a “ntop-stable.list” file which has two lines. Now you can install ntopng with:
|
1 |
sudo apt-getupdate sudo apt-getinstall ntopng |
This will install a bunch of packages, incuding ntopng, ntopng-data, pfring, redis-server, redis-tools.
Before you can start ntopng, you need to create a configuration file:
sudo nanoetcntopngntopng.conf . Read the documentation (
manntopng ) for more details. The following template can be used as a starting point:
|
1 |
—pid-path=vartmpntopng.pid —daemon —interface=eth1 —http-port=3000 —local-networks=»10.0.0.0/8,192.168.0.0/16,2001:db8::/48″ —dns-mode=1 —data-dir=vartmpntopng —disable-autologout —community |
(There can be more than one “–interface=ethX” lines in this config file if several interfaces are used in parallel.)
Furthermore, you need a file called “ntopng.start”, which can be empty but must exist in the folder:
sudo touchetcntopngntopng.start
Now you can start ntopng with:
| 1 | sudo service ntopng start |
It will also be started automatically after a reboot.
nDPI
Open and Extensible LGPLv3 Deep Packet Inspection Library.
nDPI is a ntop-maintained superset of the popular OpenDPI library. Released under the LGPL license, its goal is to extend the original library by adding new protocols that are otherwise available only on the paid version of OpenDPI. In addition to Unix platforms, we also support Windows, in order to provide you a cross-platform DPI experience. Furthermore, we have modified nDPI to be more suitable for traffic monitoring applications, by disabling specific features that slow down the DPI engine while being them un-necessary for network traffic monitoring.
nDPI is used by both ntop and nProbe for adding application-layer detection of protocols, regardless of the port being used. This means that it is possible to both detect known protocols on non-standard ports (e.g. detect http on ports other than 80), and also the opposite (e.g. detect Skype traffic on port 80). This is because nowadays the concept of port=application no longer holds.
We are continuously extending nDPI and so far many protocols are supported including
-
- FTP_CONTROL
- POP3
- SMTP
- IMAP
- DNS
- IPP
- HTTP
- MDNS
- NTP
- NetBIOS
- NFS
- SSDP
- BGP
- SNMP
- XDMCP
- SMBv1
- Syslog
- DHCP
- PostgreSQL
- MySQL
- Hotmail
- Direct_Download_Link
- POPS
- AppleJuice
- DirectConnect
- ntop
- COAP
- VMware
- SMTPS
- UBNTAC2
- Kontiki
- OpenFT
- FastTrack
- Gnutella
- eDonkey
- BitTorrent
- SkypeCall
- Signal
- Memcached
- SMBv23
- Mining
- NestLogSink
- Modbus
- Xbox
- TikTok
- RTSP
- IMAPS
- IceCast
- PPLive
- PPStream
- Zattoo
- ShoutCast
- Sopcast
- Tvants
- TVUplayer
- HTTP_Download
- QQLive
- Thunder
- Soulseek
- SSL_No_Cert
- IRC
- Ayiya
- Unencrypted_Jabber
- MSN
- Oscar
- Yahoo
- BattleField
- GooglePlus
- VRRP
- Steam
- HalfLife2
- WorldOfWarcraft
- Telnet
- STUN
- IPsec
- GRE
- ICMP
- IGMP
- EGP
- SCTP
- OSPF
- IP_in_IP
- RTP
- RDP
- VNC
- PcAnywhere
- SSL
- SSH
- Usenet
- MGCP
- IAX
- TFTP
- AFP
- Stealthnet
- Aimini
- SIP
- TruPhone
- ICMPV6
- DHCPV6
- Armagetron
- Crossfire
- Dofus
- Fiesta
- Florensia
- Guildwars
- HTTP_ActiveSync
- Kerberos
- LDAP
- MapleStory
- MsSQL-TDS
- PPTP
- Warcraft3
- WorldOfKungFu
- Slack
- Dropbox
- GMail
- GoogleMaps
- YouTube
- Skype
- DCE_RPC
- NetFlow
- sFlow
- HTTP_Connect
- HTTP_Proxy
- Citrix
- NetFlix
- LastFM
- Waze
- YouTubeUpload
- GenericProtocol
- CHECKMK
- AJP
- Apple
- Webex
- AppleiCloud
- Viber
- AppleiTunes
- Radius
- WindowsUpdate
- TeamViewer
- Tuenti
- LotusNotes
- SAP
- GTP
- UPnP
- LLMNR
- RemoteScan
- Spotify
- Messenger
- H323
- OpenVPN
- NOE
- CiscoVPN
- TeamSpeak
- Tor
- CiscoSkinny
- RTCP
- RSYNC
- Oracle
- Corba
- UbuntuONE
- Whois-DAS
- Collectd
- SOCKS
- Nintendo
- RTMP
- FTP_DATA
- Wikipedia
- ZeroMQ
- Amazon
- eBay
- CNN
- Megaco
- Redis
- Pando_Media_Booster
- VHUA
- Telegram
- Vevo
- Pandora
- QUIC
- WhatsAppVoice
- EAQ
- Ookla
- AMQP
- KakaoTalk
- KakaoTalk_Voice
- Twitch
- MPEG_TS
- Snapchat
- Sina(Weibo)
- GoogleHangout
- IFLIX
- Github
- BJNP
- SMPP
- DNScrypt
- TINC
- Deezer
- Microsoft
- Starcraft
- Teredo
- HotspotShield
- HEP
- GoogleDrive
- OCS
- Office365
- Cloudflare
- MS_OneDrive
- MQTT
- RX
- AppleStore
- OpenDNS
- Git
- DRDA
- PlayStore
- SOMEIP
- FIX
- Playstation
- Pastebin
- SoundCloud
- CSGO
- LISP
- Diameter
- ApplePush
- GoogleServices
- AmazonVideo
- GoogleDocs
- WhatsAppFiles
- Targus Dataspeed
- DNP3
- IEC60870
- Bloomberg
- CAPWAP
- Zabbix
- s7comm
- Microsoft Teams
- WebSocket
Handling Encrypted Content
The trend of Internet traffic is going towards encrypted content often using SSL. In order to let nDPI support encrypted connections, we have added a decoder for SSL (both client and server) certificates, thus we can figure out the protocol using the encryption certificate. This allows us to identify protocols such as Citrix Online and Apple iCloud that otherwise would be undetected.
Please Contribute!
DPI is a time-consuming activity as protocols (in particular P2P) change quite often. This means that it’s necessary to update the code from time to time and add extensions. We would encourage anyone out there to help us adding or enhancing new protocols: we will put your contributions on our SVN and make them available to everyone free of charge. In fact the main reason why we decided to go for nDPI instead of using the original library, is that the company behind OpenDPI has never replied to our offers to merge the extensions we coded onto the original source code.
Get It
nDPI is automatically downloaded when you build ntop and nProbe. However nothing prevents you from using it as a standalone DPI library. The source code can be downloaded from the download section.
nDPI is a registered trademark.
Statistics¶
Statistics page provides historical traffic statistics for the selected interface. The user can choose to filter
statistics on a protocol basis and display data in several formats (e.g., bytes, packets, flows, and so on). In
the Professional Version of ntopng, traffic for interface views in shown as stacked per physical interface.
Physical interface visualization can be toggled by clicking on the coloured dot just left of interface name.
The time series span can be adjusted by selecting values from 5 minutes up to 1 year. Moreover, drill-
down is possible by clicking on the time series itself. Every click zooms the chart in, centering the time
series around the clicked point.
In addition, time series shown can be chosen via the dropdown menu labelled ‘Time series’. For example,
it is possible to visualize all or just one protocol, traffic, packets, active hosts and flows, and so on. Ntopng
is VLAN aware, hence if several VLANs are detected, traffic is accounted also on a VLAN basis.
The Dropdown Time Series Menu in the
Statistics View of the Interface Details Page
The Statistics View of the Interface Details
Page (Professional Version)
Historical interface minute top talkers are shown on the right of the page, and get refreshed automatically
when the mouse moves over the chart.
An historical “Flows” tab is present next to the “Chart” when ntopng is started with the -F switch to export flows to MySQL. This
historical table shows flows data that have been recorded and dumped during the selected observation
period. The Professional/Enterprise version of ntopng also feature two extra tabs, namely, “Talkers” and
“Protocols” to drill down historical data by talkers and application protocols.
Sites¶
ntopng can keep track of top visited sites for any monitored local
host. Enabling the tracking of top visited sites requires preference
“Top HTTP Sites” to be set using the ntopng preferences page.
The Preferences Page to Enable Visited Sites
Once the preference has been enabled, this page will start showing
visited websites, which are shown over the two most recent 5-minute
intervals. Newly visited sites are placed under the “Current
Sites”. Every 5 minutes ntopng moves the “Current Sites” under “Last 5
Minute Sites”, and starts over with a clean “Current Sites”.
The Sites View of the Host Details Page
Column “Contacts” count the number of time a particular website has
been visited. Every time the host visit a site, the corresponding
“Contacts” counter is increased by one.
User Activities¶
ntopng keeps track of user activities that are related to system management and security, storing them as alerts.
These activities include:
- Users management: user added or remove, password changed
- Live traffic downloaded
- Traffic extraction (live or PCAP downloaded)
- Host JSON downloaded
- Flows data downloaded
- Alerts settings changes: alerts disabled or enabled, alerts data deleted
- Failed or successful logins
- Remote assistance enabled or disabled
- Traffic recording enabled or disabled
- Hosts/interfaces data deleted
- SNMP device added or removed
User activities are available in the , under the Past Alerts page:
PF_RING (5)
I have a DNA license, can I use it also for PF_RING ZC?
No, PF_RING ZC licenses are different from DNA licenses. You can buy a ZC license for your formed DNA license using a song discount (-50% off). Please contact us if interested.
How to capture from a bond interface using PF_RING ZC?
Since PF_RING ZC is a kernel-bypass technology and the application directly access the network card, it is not possible to capture from a bond device, however you can aggregate traffic from multiple interfaces directly in ZC, see for example zbalance_ipc … Continue reading →
ZC is faster than DNA because ZC does no copy, but DNA does?
Performance is the same, ZC provides more flexibility working in Zero-Copy (sw queues, multi-process and multi-vm support, etc.)
What is the PF_RING ZC distribution format?
PF_RING ZC is made of two components: kernel drivers and user-space library. The kernel driver is released in source format as part of PF_RING, whereas the user-space library is released in binary format and it requires a per-MAC licenses.
Applications¶
Applications page provides three pie charts and a specific table with nDPI-detected protocols for the selected
interface.
In the two top pie charts ntopng shows the application distribution and its categorization. The bottom pie
chart shows nDPI-detected applications for currently active flows. All labels are clickable and point to
detailed statistics pages. Belo pie charts there is a list of protocols detected with the corresponding total
traffic, both in absolute terms and as a percentage of the total traffic.
By selecting any Application Protocol, it is possible to display a statistics page with temporal charts for that
protocol. Similarly, by clicking on the magnifying lens icon, it is possible to display all active flows for that
protocol.
Custom Applications¶
In the Applications tab it’s possible to configure the category associated associated
with a particular application.
Applications Configuration Page
ntopng determines the application of a flow via some rules into nDPI. However,
some additional rules can be specified by the user via the –ndpi-protocols|-p <file>
option. The file has the following format:
# host:"<value>",host:"<value>",.....@<subproto> host"googlesyndacation.com"@Google host"venere.com"@Venere
An example for this configuration file is available here.
Ntopng also supports editing such file from the GUI. In order to do so, it’s
still necessary to use the above option to point to a protos file located into
a directory where ntopng has the permission to read and write. In practice, in
order to enable this feature, ntopng should be started with the
–ndpi-protocols=/var/lib/ntopng/protos.txt option. If you already have a protos
file, move it to /var/lib/ntopng/protos.txt and run
sudo chown ntopngntopng varlibntopngprotos.txt
to prepare it to be used by ntopng.
After this feature is enabled, by clicking on the “Edit Rules” button it will be
possible edit an application rules directly from the GUI and add new protocols.
It’s important to note that rules and new protocols will only be created and applied
after a restart of ntopng.
Editing Application Rules
Restore Configuration¶
ntopng configuration can be restored by placing the compressed tarball
(downloaded via Backup Configuration) into the ntopng data directory
and restarting the service. The tarball must be named conf.tar.gz.
For example, assuming the default /var/lib/ntopng data directory has not been changed, one can
restore a previously backed-up configuration by placing the compressed
tarball in /var/lib/ntopng/conf.tar.gz and then issuing a
.
Note
After the restore, the compressed archive into the data directory
is automatically deleted.
Note
Restore is only supported for packaged ntopng installations on
systems that use . Geeks and those who are building
ntopng from source should have a look at script .
TLS¶
The TLS View of the Host Details Page
ntopng registers the JA3 TLS fingerprints of an host (either when the host is a client or a server) along with their the number of uses.
As explained in the , this information is very important in order to uncover possible threats on the encrypted
traffic. By clicking on the signature it is possible to manually check if the JA3 signature
corresponds to a known malware into the abuse.ch database.
By taking a snapshot of the signatures registered on an host it is possible to know, for example,
if the host has installed a new program or has updated an existing one. This may be very important
on environments where the programs installed into an host are not supposed to change.
nProbe (4)
How to report a bug on ntop apps?
Unfortunately it can happen that our apps have a bug that leads them to crash. This note explains how to generate crash information that can be used to reproduce and resolve the issue. Whenever you have a bug to report, … Continue reading →
How do I set the input and output interface Id?
By default nProbe tries to emulate a switched environment even if a mirror packet stream is used. For this reason both input and output interfaces are set to the last two bytes of the MAC address of the packet that … Continue reading →
Recommended nProbe Templates
This post explains how to use nProbe templates for monitoring various things included HTTP traffic, and network latency.
How much bandwidth does NetFlow take during export?
Exporting traffic data in NetFlow takes some bandwith that can be pre-computed using various methods. Below you can find two calculators you can find to estimate it: Flows-per-second Estimator NetFlow Bandwidth Calculator
Payments (2)
I have problems with payments on the e-shop: what can I do?
Payments on the e-shop use PayPal. If you see the error GetExpressCheckoutDetails API call failed. Detailed Error Message: This transaction couldn’t be completed. Please redirect your customer to PayPal. Short Error Message: This transaction couldn’t be completed. Error Code: 10486 … Continue reading →
Do you offer money-back guarantee for commercial licenses?
Before purchasing commercial licenses, you have the ability to test the products to make sure that they fit your needs. All commercial products runs without license for limited time, and in case you need a longer testing period you can … Continue reading →
View category→
ntop (5)
How to report a bug on ntop apps?
Unfortunately it can happen that our apps have a bug that leads them to crash. This note explains how to generate crash information that can be used to reproduce and resolve the issue. Whenever you have a bug to report, … Continue reading →
Migrate the data directory in ntopng/nEdge
Up to release 3.6 ntopng on Unix systems is using ‘/var/tmp/ntopng’ as default directory for storing historical data, and ‘nobody‘ as default user. Since ntopng 3.7, for security reasons, the default setting for the data directory has been changed to … Continue reading →
What is the competitive advantage of ntop?
In a globalized world where components coming from many countries need to be packaged together, ntop is unique as the products are home-grown with no dependencies on third parties. This means fast application development/customization and support directly from the source. … Continue reading →
Starting ntopng on the correct network interface under Windows
ntopng for Windows is a service The ntop installer stores an empty set of options into the registry. Users who need to modify those options can: Open a shell and test your options. In order to do this type «ntop … Continue reading →
HowTo Join Public ntop Discussions
In order to join public discussion channels you need first to install discord or connect to it using a web browser. Then you need to create a discord account for logging into discord. Once there you need to join the … Continue reading →
Licensing¶
The Community edition does not need any license. Professional and Enterprise
versions require a license. ntopng automatically switches to one of these four versions,
depending on the presence of a license.
License is per-server and is released according to the EULA (End User
License Agreement). Each license is perpetual (i.e. it does not
expire) and it allows to install updates for one year since
purchase/license issue. This means that a license generated on
1/1/2021 will be able to activate new versions of the software until
12/31/2021. If you want to install new versions of the software release
after that date, you need to renew the maintenance or avoid further
updating the software. For source-based ntopng you can refer to the
GPL-v3 License.
Note
if you are using a VM or you plan to move licenses often, and you
have installed the software on a server with Internet access, you
can add to the application command
line (example: ) so
that at startup the license is validated against the license
database. The option also supports
http proxy setting the environment variable
(example: ).
Once the license has been generated, it can be applied to ntopng
simply by visiting page “Help”->”About” of the web GUI and
pasting the license key in the license form.
Alternatively, the license key can be placed in a one-line file
:
- On Linux, the file must be placed in
- On Windows, the file must be placed in
nBox
An Embedded NetFlow v5/v9/IPFIX Probe (IPv4, IPv6, MPLS)
Many software netflow solutions (such as nProbe) users, realized that running a probe on a PC (or any other common hardware) is not always the best choice for several reasons:
- PCs have moving parts that can break making the probe unavailable.
- PCs are large, need monitors and keyboards, whereas probes often need to be deployed on places where there is not much space available.
- Administering PCs is not cheap and they require the purchase of an OS, its installation and maintenance.
- In large networks divided in several trunks it is necessary to have several probes each analyzing a trunk.
This requires that multiple hosts running nProbe are deployed across the network. - The cost (for both hardware and maintenance) is not neglectable in particular if several probes need to be deployed.
If you’re a user that does not want to bother with installing software probes on a PC or that wants to have an hardware-based, scalable and purpose-built solution then you are probably a nBox user where you have a low-cost small factor computer with no moving parts and nProbe on board easy to administer using the embedded web interface.
If you want to know more in detail how the nBox works from user’s prespective, you can go through the documentation section.
The following figures shows you the nBox NetFlow/IPFIX web user interface.
This slideshow requires JavaScript.
Available Models
The nBox is available as physical appliance in various models.
| nBox | Form Factor | Monitoring Capabilities | Monitoring Port Options | Software |
|---|---|---|---|---|
| Entry Level | Desktop | Up 100 Mbit | ntopng | |
| Mini | Desktop | Up 500 Kpps | nProbe ntopng |
|
| L | 1U 19″ rackmount | Up to 2.5 Mpps |
|
PF_RING ZC nProbe Pro ntopng |
| M | 1U 19″ rackmount | Up to 6 Mpps |
|
PF_RING ZC nProbe Pro ntopng |
| H10 | 1U 19″ rackmount | Up to 10 Mpps |
|
PF_RING ZC nProbe Pro ntopng |
| H10C | 1U 19″ rackmount | Up to 14.88 Mpps | PF_RING ZC nProbe Cento ntopng |
|
| H40C | 1U 19″ rackmount | Up to 4x 14.88 Mpps |
|
PF_RING ZC nProbe Cento |
| H100C | 1U 19″ rackmount | Over 80 Mpps | PF_RING ZC nProbe Cento |
Please contact us for details, price and availability, or get in touch with our resellers.
Custom Categories¶
As shown above, ntopng already assigns a default category to the known L7 protocols.
Nevertheless, it’s also possible for the user to specify a list of additional hosts
to be included into a particular category. ntopng provides 5 empty “custom categories”
dedicated to this task, but users are also free to modify the other categories.
The custom category hosts can be specified via some host-based rules.
The host-based rules will be used to perform substring matching on some of the flow information:
If a match is found, the flow category will be set to the corresponding matching category.
These rules can be configured from the Categories tab.
By clicking “Edit Rules” it’s possible to define some rules to match hosts and associate
them to the category.
Edit Category Hosts
The picture above shows some custom hosts defined for the Web category.
Note
host matching based on IP addresses is currently limited to IPv4 flows.
Running Multiple Daemons¶
Multiple ntopng daemons can be run on the same machine when
is available. In general, this is not necessary as a
single ntopng is multi-tenant and can handle multiple
interfaces. However, there are circumstances under which it is
desirable to have multiple ntopng instances running.
To run multiple ntopng daemons, can be used. Each
daemon is identified by a so that can
be used with this identifier when controlling the daemon. For example:
systemctl start ntopng@eno1 systemctl stop ntopng@eno1 systemctl start ntopng@lo systemctl stop ntopng@lo
Each daemon must have its own configuration file under
and the configuration file name must be named as
. The example above assumes two files
and exist under
.
In order to run multiple daemons on the same machine, each daemon
must be guaranteed to have its own Redis database (option ), its
own HTTP/HTTPS ports (options and ), and its own
data directory (option ). Those options must be specified in
each daemon’s configuration file.
In order to start daemons on boot, it is necessary to enable them as
systemctl enable ntopng@eno1 systemctl enable ntopng@eno1
Daemons which have been d, will be automatically
restarted after each ntopng update. Note that backup and restore of
ntopng configuration is not supported when multiple daemons are in use.
About Us
ntop is a small, engineering-driven company, debt-free with low fixed costs, with many customers (most of which are individuals and small companies, but some key players in networking embed our software in their products), profitable in business for more than 10 years. This means that we’re potentially immune to take-overs, market decline and bankruptcy if the key customer disappears (simply because we’re lucky enough not to have a key customer). We develop high-quality networking software used by small individuals as well by large telecom operators, mostly open-source, free for no-profit/research, and provide direct support at reasonable prices. We operate our computers using the green energy (photovoltaics) we produce, our staff is geographically distributed in Europe, we mostly work from our SOHO going to the office only when necessary, this as we see a value in working close to our families and serving remote customers at night when all offices would be closed. All our software and technology is home-grown and thus independent from third-party companies, our prices are transparent and stable over time, we have full control on what we produce. Our aim is to continue operating ntop with these principles for the years to come, and not to become a sales-driven company where profit is (mostly) the only driving force, while steady growing in both revenues and staff like we have done for the past years. We like and believe in what we do, and how we do it. We expect to be here for the years to come, when you will need us.
When it all Started
ntop started as an open source project in 1998 whose goal was to create a simple yet effective web-based traffic monitoring platform. Many things have changed since then, including the nature of the traffic being analyzed, operating systems running on PCs, and the type of users. ntop changed too and from a single-project centric effort, we evolved into a full fledged research company whose goal is still the original one. Namely innovate in network monitoring using commodity hardware and open-source operating systems. As we benefit from the open-source community, we feel obliged to return to the community what we develop, so that many others, and not just us, can benefit too.
Education, No-Profit and Free Software
One of the decisions we have made from day one, is to give for free (not just a discount, but completely at no cost) our commercial software applications to universities, schools, research, no-profit and all those individuals who do not fall in these categories but that for various reasons (e.g. lost their job and are back to study for improving the skill while searching for a new one) want to use our software but cannot afford to buy it. This is our best investment: culture.
Why is ntop Different?
The industry is full of hardware companies who bundle their products with open-source products, just to offer a sale argument for their products. Or software companies who depend on other (either hardware or software companies) for running their applications. ntop does not belong to any of these categories. We are not hardware-vendor dependent, and our software does not rely on third-party companies who might impose high license fee or decide to send us (and thus our users) out of business. We are developing both software that better exploit commodity hardware (so we’re not vendor locked) and that runs on the operating system kernel, and monitoring applications. We control the whole lifecycle, optimize every single bit, and provide you support for all our components, as we have developed them and nobody knows them better than us. This is quite rare on this market, and thus we have the ability to optimize the software for the hardware we use, and create better products. Very few other companies such as Ferrari, Ducati or Apple can do that (we’re small but we believe we belong to this club).
This is why people say: ntop means open-source from the source.
As we enjoy great advantages from inventions of others, we should be glad of an opportunity to serve othersby any invention of ours. And this we should do freely and generously.
Benjamin Franklin
Automatic Daemon Startup on Boot¶
In order to launch ntopng daemon automatically on system startup, an empty file ntopng.start must be created in the same directory of the configuration files. Therefore, the directory will contain both the configuration and the startup files
root@devel:/etc/ntopng$ ls -lha total 28K drwxr-xr-x 2 root root 4.0K Mar 17 15:44 . drwxr-xr-x 117 root root 12K Mar 11 12:16 .. -rw-r--r-- 1 root root 211 Mar 15 17:54 ntopng.conf -rw-r--r-- 1 root root 0 Mar 17 15:44 ntopng.start
The existence of the file is no longer required on systems that have systemd. On those systems, automatic ntopng daemon startup is controlled by enabling/disabling the ntopng service as
Promiscuous Interfaces
What’s still missing is the configuration of the eth1 interface to be in promisc mode. Furthermore, it should not get an IPv4 or IPv6 via DHCPv4 or SLAAC. Therefore, the following configuration steps are required.
Disable IPv6 on the interface: Open the following file:
| 1 | sudo nanoetcsysctl.conf |
and add the following line:
| 1 | net.ipv6.conf.eth1.disable_ipv6=1 |
Start the eth1 interface in promiscuous mode: Open the following file:
| 1 | sudo nanoetcnetworkinterfaces |
and add these lines:
|
1 |
auto eth1 iface eth1 inet manual up ifconfig eth1 promisc up down ifconfig eth1 promisc down |
Note: If there are already some lines that reference to eth1, delete them or comment them out. For example, there should be no “iface eth1 inet dhcp” line anymore!
Reboot the server:
sudo reboot .
Now, after each reboot of the server, the eth1 interface card will be in promiscuous mode and ntopng will be started automatically.
To verify that ntopng is running, have a look at netstat, which should display the running process and the open TCP port 3000:
|
1 8 |
weberjoh@jw-nb10etcntopng$sudo netstat-l-p-n Active Internet connections(only servers) Proto Recv-QSend-QLocal Address Foreign Address State PIDProgram name tcp127.0.0.163790.0.0.0*LISTEN1280redis-server1 tcp0.0.0.0220.0.0.0*LISTEN1110sshd tcp6::22::*LISTEN1110sshd udp192.168.120.101230.0.0.0*1729ntpd udp127.0.0.11230.0.0.0*1729ntpd udp0.0.0.01230.0.0.0*1729ntpd udp0.0.0.01610.0.0.0*1307snmpd udp0.0.0.0588200.0.0.0*1307snmpd udp0.0.0.05140.0.0.0*1236syslog-ng udp62003516012120::1123::*1729ntpd udp6fe80::21d92fffe53123::*1729ntpd udp6::1123::*1729ntpd udp6::123::*1729ntpd udp6::1161::*1307snmpd |
Using Behind a Firewall¶
In the remainder of this section it is shown how to connect nProbe and ntopng in presence of a NAT or firewalls. Indeed, the examples given above might not have worked well in case there was a firewall or a NAT between nProbe and ntopng. Following it is shown an exhaustive list of all the possible scenarios that may involve firewalls or NATs, and the configuration that has to be used to always ensure connectivity between nProbe and ntopng.
nProbe and ntopng on the same private network (firewall protected)
In this scenario, the firewall does not create any trouble to ZMQ communications and the normal configurations described above can be used.
nProbe on a public network/IP, ntopng on a private network/IP protected by a firewall
In this case the ZMQ paradigm works well as ntopng connects to nProbe and the normal configurations highlighted above can be used.
nProbe on a private network/IP, ntopng on a public network/IP protected by a firewall
In this case the ZMQ paradigm does not work as the firewall prevents ntopng (connection initiator) to connect to nProbe. In this case it is necessary to revert the ZMQ paradigm by swapping the roles of nProbe and ntopng. Suppose nProbe runs on host and ntopng on host . In this scenario it is necessary to start the applications as follows
nprobe --zmq-probe-mode --zmq "tcp://46.101.x.y:5556" -i eth1 -n none ntopng -i "tcp://*:5556c"
Note the two options:
- tells nProbe to initiate a connection to
- tells ntopng to act as a collector (notice the small ) and to listen for incoming connections.
In essence the roles of nProbe and ntopng have been reverted so they behave as NetFlow/IPFIX probes do. Only the roles have been reverted. Everything else will continue to work normally and the flows will still go from nProbe to ntopng.
Home¶
Home is the default view of the Host Details page and provides detailed information including host MAC
Address (or the last router MAC address if the host is remote), IP Address (with network mask if detected),
a toggle to activate/deactivate alerts for the host, a checkbox to enable packet dump for the specific host,
symbolic hostname (or IP address), location (local or remote), date and time of first and last packet seen
for the host, traffic breakdown, amount of traffic packets received/sent, number of flows as client/server
host. All of this information is also available in JSON format by clicking on the ‘Download’ link. The heat
map provides the Activity Map for each host. Each box represents one minute of traffic. By default, Activity
Map shows the last six hours, but it is possible to set a different timeframe using the controls.
