I’ve been working with the radanalytics.io big data examples for OpenShift recently and every once in a while I would be on a slow network and plagued with inconsistency in deploys to get the entire example running. I finally reached out for help and got some great debugging advice so I wanted to share some of the basics on how to tell what is going on when a deployment just isn’t finishing and you aren’t getting much information about it.
Here was my scenario. I was running through the Value at Risk example and sometimes the Oshinko Web UI wouldn’t deploy properly. Sometimes I would get past that but then the Spark containers wouldn’t deploy. Looking in the deployment logs didn’t really help as I would only see something like:
--> Scaling sparky-m-1 to 1
--> Waiting up to 10m0s for pods in rc sparky-m-1 to become ready
The trick was that I had to figure out what was actually happening in that deployment. The first step was to run oc get pods
to find the non-deployment pod:
[mhicks@localhost bigdata]$ oc get pods
NAME READY STATUS RESTARTS AGE
oshinko-1-73jg7 1/1 Running 0 17m
sparky-m-1-deploy 1/1 Running 0 7m
sparky-m-1-r1vvj 0/1 ContainerCreating 0 7m
sparky-w-1-2qg0m 0/1 ContainerCreating 0 7m
sparky-w-1-deploy 1/1 Running 0 7m
I’ve bolded the two pods in this example.
Next, I needed to figure out what was happening in those deployments. You can get this from the oc describe <pod>
command if you look in the events section (clipped output below):
[mhicks@localhost bigdata]$ oc describe pod sparky-m-1-r1vvj
Name: sparky-m-1-r1vvj
Namespace: myproject
...
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
8m 8m 1 {default-scheduler } Normal Scheduled Successfully assigned sparky-m-1-r1vvj to 192.168.10.222
8m 8m 1 {kubelet 192.168.10.222} spec.containers{sparky-m} Normal Pulling pulling image "willb/var-spark-worker"
Interesting… Check out that last event with pulling image “willb/var-spark-worker”. That means that it’s still doing a docker pull.
The last step is to be able to check the progress of that docker pull. That’s simple enough by actually running docker pull <image>
on the same image.
[mhicks@localhost bigdata]$ docker pull willb/var-spark-worker
Using default tag: latest
Trying to pull repository docker.io/willb/var-spark-worker ...
sha256:70a5248e91444b96c66d0555df23c41938a7ae68e16941ee47f8ce3ed49a965a: Pulling from docker.io/willb/var-spark-worker
8d30e94188e7: Already exists
b4cef18dbaf6: Already exists
67005339c478: Downloading [=============================> ] 110.8 MB/187.5 MB
4c505a838158: Download complete
28001ba6816a: Download complete
6f9875b2f6b6: Downloading [==========> ] 38.91 MB/187.5 MB
a0ccab00fadc: Download complete
9d123a390bac: Downloading [===============> ] 13.72 MB/44.25 MB
4869f3d7d89e: Waiting
a25f81ddacf4: Waiting
ff249abd99d4: Waiting
And there you have it. Now you can not only know what is holding up your deployment but you can also track the progress to really know when it’s done.
Hope this helps!
저자 소개
Matt Hicks was named President and Chief Executive Officer of Red Hat in July 2022. In his previous role, he was Executive Vice President of Products and Technologies where he was responsible for product engineering for much of the company’s portfolio, including Red Hat® OpenShift® and Red Hat Enterprise Linux®. He is one of the founding members of the OpenShift team and has been at the forefront of cloud computing ever since.
Prior to joining Red Hat 16 years ago, Hicks served in various roles spanning computer engineering, IT, and consulting. He has worked with Linux and open source for more than 25 years, and his breadth of experience has helped him solve customer and business problems across all areas of IT.
채널별 검색
오토메이션
기술, 팀, 인프라를 위한 IT 자동화 최신 동향
인공지능
고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트
오픈 하이브리드 클라우드
하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요
보안
환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보
엣지 컴퓨팅
엣지에서의 운영을 단순화하는 플랫폼 업데이트
인프라
세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보
애플리케이션
복잡한 애플리케이션에 대한 솔루션 더 보기
오리지널 쇼
엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리
제품
- Red Hat Enterprise Linux
- Red Hat OpenShift Enterprise
- Red Hat Ansible Automation Platform
- 클라우드 서비스
- 모든 제품 보기
툴
체험, 구매 & 영업
커뮤니케이션
Red Hat 소개
Red Hat은 Linux, 클라우드, 컨테이너, 쿠버네티스 등을 포함한 글로벌 엔터프라이즈 오픈소스 솔루션 공급업체입니다. Red Hat은 코어 데이터센터에서 네트워크 엣지에 이르기까지 다양한 플랫폼과 환경에서 기업의 업무 편의성을 높여 주는 강화된 기능의 솔루션을 제공합니다.