Ask Your Question
0

[python] RAM usage high when using VideoWriter in jupyter ONLY not pycharm

asked 2020-05-18 11:05:26 -0600

meathead2k2 gravatar image

updated 2020-05-18 11:37:40 -0600

I'm writing around 1200 images to video with opencv's videowriter.

When running in pycharm my ram stays at around 6gb used.

When I run exactly the same code in jupyter my ram slowly increases and by the time we are nearly at 1200 frames the ram is maxed out and sometimes the computer completely hangs.

I've tired different codecs and they all have the same effect.

I've tried deleting the VideoWriter object after the rendering is complete and the RAM usage does not clear.

I've tried garbage collection and no effect on ram.

Restarting the jupyter kernel DOES clear the RAM.

Does anyone have any experience with this or can suggest any solutions?

EDIT:

Self hosted jupyter!

General configuration for OpenCV 4.2.0 ===================================== Version control: 4.2.0 Platform: Timestamp: 2020-04-04T14:38:39Z Host: Linux 4.15.0-1028-gcp x86_64 CMake: 3.9.0 CMake generator: Unix Makefiles CMake build tool: /usr/bin/gmake Configuration: Release CPU/HW features: Baseline: SSE SSE2 SSE3 requested: SSE3 Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX SSE4_1 (14 files): + SSSE3 SSE4_1 SSE4_2 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 (0 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX AVX (4 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX C/C++: Built as dynamic libs?: NO C++ Compiler: /usr/lib/ccache/compilers/c++ (ver 4.8.2) C++ flags (Release): -Wl,-strip-all -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-comment -Wno-missing-field-initializers -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG C++ flags (Debug): -Wl,-strip-all -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-comment -Wno-missing-field-initializers -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG C Compiler: /usr/lib/ccache/compilers/cc C flags (Release): -Wl,-strip-all -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wuninitialized -Winit-self -Wno-comment -Wno-missing-field-initializers -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG C flags (Debug): -Wl,-strip-all -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wuninitialized -Winit-self -Wno-comment -Wno-missing-field-initializers -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG Linker flags (Release): -L/root/ffmpeg_build/lib -Wl,--gc-sections
Linker flags (Debug): -L/root/ffmpeg_build/lib -Wl,--gc-sections
ccache: YES Precompiled headers: NO Extra dependencies: ade /lib64/libgthread-2.0.so /lib64/libglib-2.0.so /lib64/libz.so dl m pthread rt 3rdparty dependencies: ittnotify libprotobuf libjpeg-turbo libwebp libpng libtiff libjasper IlmImf quirc OpenCV modules: To be built: calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio Disabled: world Disabled by dependency: - Unavailable: java js python2 ts Applications: - Documentation: NO ... (more)

edit retag flag offensive close merge delete

Comments

please show code, dear. also -- opencv version / from where / os -- whatever you can give us

code in jupyter

that can be anything. self hosted ? some internet platform, like colab ?

berak gravatar imageberak ( 2020-05-18 11:11:47 -0600 )edit
1

will add to the post

meathead2k2 gravatar imagemeathead2k2 ( 2020-05-18 11:34:14 -0600 )edit
1

linux kubuntu 20.04

meathead2k2 gravatar imagemeathead2k2 ( 2020-05-18 11:52:44 -0600 )edit

@berak Any idea?

meathead2k2 gravatar imagemeathead2k2 ( 2020-05-19 00:58:26 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
-1

answered 2020-05-19 04:25:46 -0600

meathead2k2 gravatar image

@berak It wont let me post the answer as im a "new user" but its matplotlibs fault.

I just need to call plt.ioff() ARGHH

edit flag offensive delete link more

Comments

as im a "new user"

sure we can help !

berak gravatar imageberak ( 2020-05-19 04:47:00 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2020-05-18 11:05:26 -0600

Seen: 611 times

Last updated: May 18 '20