详解Docker无法正常启动的原因及解决办法
一、Docker启动异常表现:
1.状态反复restaring,用命令查看
$dockerps-a CONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES 21c09be88c11docker.xxxx.cn:5000/xxx-tes/xxx_tes:1.0.6"/usr/local/tomcat..."9daysagoRestarting(1)Lessthanasecondagoxxx10
2.Docker日志有明显问题:
$dockerlogs[容器名/容器ID]
二、Docker启动异常的可能原因:
2.1.内存不够
Docker启动至少需要2G内存,首先执行free-mh命令查看剩余内存是否足够
直接查看内存
$free-mh totalusedfreesharedbuff/cacheavailable Mem:15G14G627M195M636M726M Swap:0B0B0B
分析日志
有时候一瞬间内存过载溢出,导致部分进程被杀死,看起来内存也是够用的,事实上docker还是会反复重启,就需要通过docker日志和系统日志信的息来进一步分析:
分析docker日志
查看docker日志看到内存溢出的信息,要仔细翻阅才能找到信息,并不是在最下面
$dockerlogs[容器名/容器ID]|less JavaHotSpot(TM)64-BitServerVMwarning:INFO:os::commit_memory(0x0000000769990000,1449590784,0)failed;error='Cannotallocatememory'(errno=12) # #ThereisinsufficientmemoryfortheJavaRuntimeEnvironmenttocontinue. #Nativememoryallocation(malloc)failedtoallocate1449590784bytesforcommittingreservedmemory. #Anerrorreportfilewithmoreinformationissavedas: #//hs_err_pid1.log JavaHotSpot(TM)64-BitServerVMwarning:INFO:os::commit_memory(0x0000000769990000,1449590784,0)failed;error='Cannotallocatememory'(errno=12) # #ThereisinsufficientmemoryfortheJavaRuntimeEnvironmenttocontinue. #Nativememoryallocation(malloc)failedtoallocate1449590784bytesforcommittingreservedmemory. #Anerrorreportfilewithmoreinformationissavedas: #/tmp/hs_err_pid1.log JavaHotSpot(TM)64-BitServerVMwarning:INFO:os::commit_memory(0x0000000769990000,1449590784,0)failed;error='Cannotallocatememory'(errno=12) # #ThereisinsufficientmemoryfortheJavaRuntimeEnvironmenttocontinue. #Nativememoryallocation(malloc)failedtoallocate1449590784bytesforcommittingreservedmemory. #Cannotsavelogfile,dumptoscreen.. # #ThereisinsufficientmemoryfortheJavaRuntimeEnvironmenttocontinue. #Nativememoryallocation(malloc)failedtoallocate1449590784bytesforcommittingreservedmemory. #Possiblereasons: #ThesystemisoutofphysicalRAMorswapspace #In32bitmode,theprocesssizelimitwashit #Possiblesolutions: #Reducememoryloadonthesystem #Increasephysicalmemoryorswapspace #Checkifswapbackingstoreisfull #Use64bitJavaona64bitOS #DecreaseJavaheapsize(-Xmx/-Xms) #DecreasenumberofJavathreads #DecreaseJavathreadstacksizes(-Xss) #Setlargercodecachewith-XX:ReservedCodeCacheSize= #Thisoutputfilemaybetruncatedorincomplete. # #OutofMemoryError(os_linux.cpp:2756),pid=1,tid=140325689620224 # #JREversion:(7.0_79-b15)(build) #JavaVM:JavaHotSpot(TM)64-BitServerVM(24.79-b02mixedmodelinux-amd64compressedoops) #Coredumpwritten.Defaultlocation://coreorcore.1 #
分析系统日志
查看系统日志,发现有大量由于内存溢出,进程被杀死的记录
$grep-i'OutofMemory'/var/log/messages Apr710:04:02centos106kernel:Outofmemory:Killprocess1192(java)score54orsacrificechild Apr710:08:00centos106kernel:Outofmemory:Killprocess2301(java)score54orsacrificechild Apr710:09:59centos106kernel:Outofmemory:Killprocess28145(java)score52orsacrificechild Apr710:20:40centos106kernel:Outofmemory:Killprocess2976(java)score54orsacrificechild Apr710:21:08centos106kernel:Outofmemory:Killprocess3577(java)score47orsacrificechild Apr710:21:08centos106kernel:Outofmemory:Killprocess3631(java)score47orsacrificechild Apr710:21:08centos106kernel:Outofmemory:Killprocess3634(java)score47orsacrificechild Apr710:21:08centos106kernel:Outofmemory:Killprocess3640(java)score47orsacrificechild Apr710:21:08centos106kernel:Outofmemory:Killprocess3654(java)score47orsacrificechild Apr710:27:27centos106kernel:Outofmemory:Killprocess6998(java)score51orsacrificechild Apr710:27:28centos106kernel:Outofmemory:Killprocess7027(java)score52orsacrificechild Apr710:28:10centos106kernel:Outofmemory:Killprocess7571(java)score42orsacrificechild Apr710:28:10centos106kernel:Outofmemory:Killprocess7586(java)score42orsacrificechild
2.2.端口冲突
该docker监听端口已经被其他进程占用,一般此种问题容易出现在新部署的服务,或在原有机器上部署新的后台服务,所以在部署之前应该执行命令检查端口是否已经被占用,如果上线后发现占有则应改为可用端口再重启之。
检查命令: $netstat-nltp|grep[规划的端口号]
三、对策
3.1.内存不够的对策:
对策1:
3.1.1saltstack的minion在运行过久之后,可能占用大量内存,需要将其重启。重启命令可能有时并不起作用。主要检查运行状态,如果未成功停止,则重新重启;
对策2:
3.2.2ELK日志收集程序或者其他java进程占用过高,用top和ps命令排查,谨慎确定进程的作用,在确保不影响业务的情况下,停止相关进程;
对策3:
释放被占用的内存(buff/cache):
$sync #将内存数据写入磁盘
$echo3>/proc/sys/vm/drop_caches #释放被占用的内存
对策4:
有时候并不是buff/cache过高导致内存不够用,确实是被很多必要的进程消耗掉了内存,那就需要从机器资源分配使用的层面去考虑和解决了。
3.2端口冲突的对策
对策1:
一般此种问题容易出现在新部署的服务,或在原有机器上部署新的后台服务,所以在部署之前应该执行命令检查端口是否已经被占用,如果上线后发现占有则应改为可用端口再重启之。
检查命令: $netstat-nltp|grep[规划的端口号]
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持毛票票。