基于alpine用dockerfile创建的爬虫Scrapy镜像的实现

2023-09-04 17:54:04 65

一、下载alpine镜像

[root@DockerBrian~]#dockerpullalpine
Usingdefaulttag:latest
Tryingtopullrepositorydocker.io/library/alpine...
latest:Pullingfromdocker.io/library/alpine
4fe2ade4980c:Pullcomplete
Digest:sha256:621c2f39f8133acb8e64023a94dbdf0d5ca81896102b9e57c0dc184cadaf5528
Status:Downloadednewerimagefordocker.io/alpine:latest
[root@docker43~]#dockerimages
REPOSITORYTAGIMAGEIDCREATEDSIZE
docker.io/alpinelatest196d12cf6ab13weeksago4.41MB

二、编写Dockerfile

创建scrapy目录存放dockerfile文件

[root@DockerBrian~]#mkdir/opt/alpineDockerfile/
[root@DockerBrian~]#cd/opt/alpineDockerfile/
[root@DockerBrianalpineDockerfile]#mkdirscrapy&&cdscrapy&&touchDockerfile
[root@DockerBrianalpineDockerfile]#cdscrapy/
[root@DockerBrianscrapy]#ll
总用量4
-rw-r--r--1rootroot139410月1011:36Dockerfile

编写dockerfile文件

#指定创建的基础镜像
FROMalpine

#作者描述信息
MAINTAINERalpine_python3_scrapy(zhujingzhi@123.com)

#替换阿里云的源
RUNecho"http://mirrors.aliyun.com/alpine/latest-stable/main/">/etc/apk/repositories&&\
echo"http://mirrors.aliyun.com/alpine/latest-stable/community/">>/etc/apk/repositories

#同步时间

#更新源、安装openssh并修改配置文件和生成key并且同步时间
RUNapkupdate&&\
apkadd--no-cacheopenssh-servertzdata&&\
cp/usr/share/zoneinfo/Asia/Shanghai/etc/localtime&&\
sed-i"s/#PermitRootLogin.*/PermitRootLoginyes/g"/etc/ssh/sshd_config&&\
ssh-keygen-trsa-P""-f/etc/ssh/ssh_host_rsa_key&&\
ssh-keygen-tecdsa-P""-f/etc/ssh/ssh_host_ecdsa_key&&\
ssh-keygen-ted25519-P""-f/etc/ssh/ssh_host_ed25519_key&&\
echo"root:h056zHJLg85oW5xh7VtSa"|chpasswd

#安装Scrapy依赖包(必须安装的依赖包)
RUNapkadd--no-cachepython3python3-devgccopenssl-devopenssllibressllibc-devlinux-headerslibffi-devlibxml2-devlibxml2libxslt-devopenssh-clientopenssh-sftp-server

#安装环境需要pip包(这里的包可以按照需求添加或者删除)
RUNpip3install--default-timeout=100--no-cache-dir--upgradepipsetuptoolspymysqlpymongoredisscrapy-redisipythonScrapyrequests

#启动ssh脚本
RUNecho"/usr/sbin/sshd-D">>/etc/start.sh&&\
chmod+x/etc/start.sh

#开放22端口
EXPOSE22

#执行ssh启动命令
CMD["/bin/sh","/etc/start.sh"]

实现了容器可以SSH远程访问基于Python3环境安装的Scrapy，通过start.sh脚本启动SSH服务

三、创建镜像

创建镜像

[root@DockerBrianscrapy]#dockerbuild-tscrapy_redis_ssh:v1.

查看镜像

[root@DockerBrianscrapy]#dockerimages
REPOSITORYTAGIMAGEIDCREATEDSIZE
scrapy_redis_sshv1b2c95ef95fb94hoursago282MB
docker.io/alpinelatest196d12cf6ab14weeksago4.41MB

四、创建容器

创建容器(名字为scrapy10086远程端口是映射宿主机10086端口)

dockerrun-itd--restart=always--namescrapy10086-p10086:22scrapy_redis_ssh:v1

查看容器

[root@DockerBrianscrapy]#dockerps
CONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES
7fb9e69d79f5b2c95ef95fb9"/bin/sh/etc/star..."3hoursagoUp3hours0.0.0.0:10086->22/tcpscrapy10086

登录容器

[root@DockerBrianscrapy]#sshroot@127.0.0.1-p10086
Theauthenticityofhost'[127.0.0.1]:10086([127.0.0.1]:10086)'can'tbeestablished.
ECDSAkeyfingerprintisSHA256:wC46AU6SLjHyEfQWX6d6ht9MdpGKodeMOK6/cONcpxk.
ECDSAkeyfingerprintisMD5:6a:b7:31:3c:63:02:ca:74:5b:d9:68:42:08:be:22:fc.
Areyousureyouwanttocontinueconnecting(yes/no)?yes
Warning:Permanentlyadded'[127.0.0.1]:10086'(ECDSA)tothelistofknownhosts.
root@127.0.0.1'spassword:#这里的密码就是dockerfile中定义的echo"root:h056zHJLg85oW5xh7VtSa"|chpasswd
WelcometoAlpine!

TheAlpineWikicontainsalargeamountofhow-toguidesandgeneral
informationaboutadministratingAlpinesystems.
See.

Youcansetupthesystemwiththecommand:setup-alpine

Youmaychangethismessagebyediting/etc/motd.

7363738cc96a:~#

五、测试

创建个scrapy项目测试

7363738cc96a:~#scrapystartprojecttest
NewScrapyproject'test',usingtemplatedirectory'/usr/lib/python3.6/site-packages/scrapy/templates/project',createdin:
/root/test

Youcanstartyourfirstspiderwith:
cdtest
scrapygenspiderexampleexample.com
7363738cc96a:~#cdtest/
7363738cc96a:~/test#ls
scrapy.cfgtest
7363738cc96a:~/test#cdtest/
7363738cc96a:~/test/test#ls
__init__.py__pycache__items.pymiddlewares.pypipelines.pysettings.pyspiders
7363738cc96a:~/test/test#

测试成功

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持毛票票。

基于alpine用dockerfile创建的爬虫Scrapy镜像的实现

热门推荐

随机推荐