RFlow

RFlow

Pipe-able task manager for routine server maintenance, specially crafted towards management of blog networks. With extra stuff for WordPress.

TheoryDigital for Originally for EHDA, vastly expanded.

Timeframe: 2016+

Linux system maintenance, Blog network management tools, PBN maintenance, WordPress

Tools: LAMP/LEMP, Ruby, Bash

What it is: Task manager for server maintenance. Written in Ruby. It has particular focus on WordPress and maintenance of blog networks.

What it does: Shifts time expenditures from routine tasks without complicated setups. Pays off even for smaller networks.

How it works: The master branch of RFlow is to be installed and conf’d on your servers. The RPC branch is to run an orchestrated sequence that pulls all working data from your servers into your laptop. This way you can browse all access log data for your whole network at a single run.

Below is a config example for the RPC branch.

  • You define your server network and access data, one server can be configured as bastion.
  • If there is a bastion, RFlow will access the rest of your servers through it, so that you can deny SSH access from other places than your bastion host.
  • In an omitted part you set up hooks for alerting via Slack.
# Rflow RPC conf
env:         'tf-local'
root:        '/home/theoryforce/rflow'
sdb:         '/tmp/rflow/state'
clients:     [client1, client2, db, dev, bastion]
servermap:
  client1:   [uname, xxx.xxx.xxx.xxx]
  client2:   [uname, xxx.xxx.xxx.xxx]
  db:        [uname, xxx.xxx.xxx.xxx]
  dev:       [uname, xxx.xxx.xxx.xxx]
  bastion:   [uname, xxx.xxx.xxx.xxx]

domains:     ['example.com', 'some.tech']
bastion:
  server:    'xxx.xxx.xxx.xxx'
  user:      'uname'
  code:      'bastion'
  key:       'id_ed25519_bastion'
servergroup:
  production:   db, client1
  hilit:        client2

api:
  neutrino:
...

You can still run pipes on clients as well.

Managing everything locally is less convenient for some parts or the routine maintenance. Most of these on-client tasks are updates and reports though - in other words, processes that don’t require your attention. They can be run from a crontab.

All pipes trigger a state change.

States are stored as hex data in simple string data base, one of Ruby’s default libraries.

These tiny text files are retrieved when you start an RPC pipe, so that you get a single-glance overview of all maintenance sequences that triggered in the past days.

Both RPC and client branches come with a Rakefile and a binary.

rflow -- RFlow is a sysadmin pipeline system in Ruby with some Bash scripts included because we are not too anal about nice code here.

Usage:

  rflow <subcommand> [options]

Options:
        -h, --help         Show this message
        -v, --version      Print the name and version
        -t, --trace        Show the full backtrace when an error occurs

Subcommands:
  bastion, via          Runs flow on client servers through a bastion server. Which server is a bastion needs to be configured.
  state                 Retrieves state note temps from family.
  auth, a               Runs the orchestration for pulling and redistributing bad hits of client authlogs.
  access, l             Runs the orchestration for access log scan.
  blacklists, bl, b     Slurps client blacklists and redistributes the slurp.
  techseo               Pulls TechSEO reports from sibling servers into a CSV. (Needs counterparts on client)
  infosec               Nmap network scanning and server log backups for configured clients.
  loadtime, lt          Runs the curl loadtime report for submitted domain (Assumes HTTPS).
  wpscan, wp            Runs the wpscan for a domain (Assumes HTTPS), --csv as an option.
  hallow, ha            [util] Check if current IP is whitelisted on all servers and if not, add it to hosts.allow
  ip                    [util] Display your public IP
  probe                 [util] Neutrino API probe for IP

You can also quickly add your own pipes as you go, as child classes of RFlow.

Written in Ruby and Bash.
Other Work

Publishing Network

Digital Marketing

RECOVID

Information Architecture

RFlow

Backend