使用Python五分钟搭建100台服务器squid环境

前言

在日常工作中，我们要搭建linux环境，要么自己登陆服务器输入命令安装，要么使用docker安装。但这种方式对于有大量机器时候就显得力不从心了。所以，这篇文章要讲的就是以squid为例子使用python批量部署服务器环境。

准备

1. 首先你要有台开机的服务器，为避免冲突，最好是新的vps没有安装任何服务。
2. 拥有root账户，因为很多服务的安装都需要管理员权限。

正本

squid

squid是一个linux代理服务软件，提供简单的代理服务，可以用在简单的代理场景，生产中常设置IP白名单限制滥用，也可以使用密码授权。建议开启Https防止裸奔。

搭建

今天出场的猪脚就是著名的 Paramiko，ssh是一个协议，OpenSSH是其中一个开源实现，paramiko是Python的一个库，实现了ssh v2 协议(底层使用cryptography)。

有了Paramiko以后，我们就可以在Python代码中直接使用SSH协议对远程服务器执行操作，而不是通过ssh命令对远程服务器进行操作。

• paramiko 用户ssh连接
• threading 用户批量操作机器

想必看到这里你已经懂了，就是利用多线程和paramiko批量操作服务器

用到的库先导入

import paramiko
import random
import string
import threading
import logging

初始化日志，方便调试

mlog = logging.getLogger("ssh")
fmt = '%(asctime)s|%(levelname)s|%(filename)s:%(lineno)d|%(message)s'
mlog.setLevel(logging.DEBUG)
formatter = logging.Formatter(fmt)  
console = logging.StreamHandler()  
console.setLevel(logging.DEBUG)
console.setFormatter(formatter)
mlog.addHandler(console)

下面来定义工作线程

class ParamikoThreading(threading.Thread):
    def __init__(self, hostname, port, username,password):
        threading.Thread.__init__(self)
        self.hostname = hostname
        self.port = port
        self.username = username
        self.password = password
        #创建一个ssh的客户端，用来连接服务器
        self.ssh = paramiko.SSHClient()
        #创建一个ssh的白名单
        know_host = paramiko.AutoAddPolicy()
        #加载创建的白名单
        self.ssh.set_missing_host_key_policy(know_host)
        self.ssh.connect(
            hostname = hostname,
            port = port,
            username = username,
            password = password
        )
        self.get_random_user()

为了系统的安全性，我们随机密码，这里你也可以利用自己的方式随机密码，或固定设置

    def get_random_user(self):
        # 生成用户名密码
        a=string.ascii_letters+string.digits
        key=random.sample(a,6)
        self.proxy_user= "user" + "".join(key)
        self.proxy_user = self.proxy_user.lower()
        a=string.ascii_letters+string.digits
        key=random.sample(a,6)
        self.proxy_passwd= "pass" + "".join(key)
        self.proxy_passwd = self.proxy_passwd.lower()

安装 squid服务并设置开启自启，这里我已经把配置文件统一写好了，使用时候直接上传就好了，简化了部署的流程

/mnt/sda/work/project/proxy_account/squid.conf

简单来说，下面的代码就是把安装环境手动敲击的命令，使用程序连续输入

    def config_env(self):
        #执行命令
        cmds = ["yum -y install squid", 
                "systemctl enable squid.service", 
                "systemctl start squid.service",
                "yum -y install httpd",
                "cd /etc/squid/; htpasswd -b -c /etc/squid/passwd  {} {}".format(self.proxy_user, self.proxy_passwd)]
        for cmd in cmds:
            stdin,stdout,stderr = self.ssh.exec_command(cmd)
            mlog.debug("execute cmd[{}] result:\n{}".format(cmd, stdout.read().decode()) )
            
            
        # 上传配置文件
        sftp = self.ssh.open_sftp()
        sftp.put("/mnt/sda/work/project/proxy_account/squid.conf", "/etc/squid/squid.conf")
        mlog.debug("put file[/mnt/sda/work/project/proxy_account/squid.conf /etc/squid/squid.conf]")
        sftp.close()
        
        
        # 重启服务
        stdin,stdout,stderr = self.ssh.exec_command("systemctl restart squid.service")
        mlog.debug("execute cmd[{}] result:\n{}".format("systemctl restart squid.service", stdout.read().decode()) )
        

最终账户密码主机等信息保存在文件中result_{hostname}.txt

        # 保存结果文件
        with open("result_{}.txt".format(self.hostname), "w") as fp:
            fp.write("http://{}:{}@{}:{}\n".format(self.proxy_user, self.proxy_passwd, self.hostname, 3389))
        # 关闭连接
        self.ssh.close()
        
    def run(self):
        mlog.info("begin host[{}]".format(self.hostname))
        self.config_env()
        mlog.info("done host[{}]".format(self.hostname))
     

上面的代码就完成了远程操作的基本定义了，然后我们来使用这个代码

p = ParamikoThreading("23.24.46.11", 22, "root", "dddd#nj3XRK8r12323")
p.setDaemon(True)
p.start()
p.join()

运行完成点击打开保存的文件，就是我们搭建的服务了，测试服务是否可用, 如果代码没有报错说明你的服务成功了

import requests
proxies = {
    "http": f"http://{user}:{password}@{host}:{port}",
    "https": f"http://{user}:{password}@{host}:{port}",
}
r = requests.get("https://httpbin.org/get",proxies=proxies)

print(r.status)
print(r.text)

使用

通过以上就完成了服务器的远程搭建，如果有100台服务器怎么办呢

ip、账户名、密码填入实际值

threads = []
for _ in range(100):
    p = ParamikoThreading("23.24.46.11", 22, "root", "dddd#nj3XRK8r12323")
    p.setDaemon(True)
    p.start()
    threads.append(p)
    
for t in threads:
    p.join()

不出意外地话，五分钟，一百台服务器全部搭建完成，就是这么简单

结尾

本文讲述使用Python批量远程ssh搭建服务，大家可以举一反三，构建自己的生产环境自动控制服务。

附 squid.conf 配置文件

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255    # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8        # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10        # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16     # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12        # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16        # RFC 1918 local private network (LAN)
acl localnet src fc00::/7           # RFC 4193 local private network range
acl localnet src fe80::/10          # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80        # http
acl Safe_ports port 21        # ftp
acl Safe_ports port 443        # https
acl Safe_ports port 70        # gopher
acl Safe_ports port 210        # wais
acl Safe_ports port 1025-65535    # unregistered ports
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 488        # gss-http
acl Safe_ports port 591        # filemaker
acl Safe_ports port 777        # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost


auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd
auth_param basic realm proxy
acl authenticated proxy_auth REQUIRED
http_access allow authenticated

# And finally deny all other access to this proxy
http_access deny all

request_header_access Via deny all
request_header_access X-Forwarded-For deny all

# Squid normally listens to port 3128
http_port 3389

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:        1440    20%    10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern -i (/cgi-bin/|\?) 0    0%    0
refresh_pattern .        0    20%    4320