With our Percona Server for MongoDB 3.2 release, we made RocksDB a first class citizen. With this newly-available engine, we want to make it easy for everybody interested to try it. So it is now available in docker images from -server-mongodb/.
RocksDB from Docker containers
---> Running in 1e42a5e50cbe Requirement already satisfied:faust[rocksdb] in /usr/local/lib/python3.10/site-packages (1.10.4)Requirement already satisfied: terminaltables=3.1 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (3.1.10)Requirement already satisfied: click=6.7 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (7.1.2)Requirement already satisfied: yarl=1.0 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (1.7.2)Requirement already satisfied: aiohttp-cors=0.7 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (0.7.0)Requirement already satisfied: mypy-extensions in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (0.4.3)Requirement already satisfied: colorclass=2.2 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (2.2.2)Requirement already satisfied: opentracing=1.3.0 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (1.3.0)Requirement already satisfied: mode=4.3.2 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (4.3.2)Requirement already satisfied: venusian=1.1 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (1.2.0)Requirement already satisfied: aiohttp=3.5.2 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (3.8.1)Requirement already satisfied: robinhood-aiokafka=1.1.6 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (1.1.6)Requirement already satisfied: croniter>=0.3.16 in/usr/local/lib/python3.10/site-packages (from faust[rocksdb]) (1.1.0)Collecting python-rocksdb>=0.6.7 Downloadingpython-rocksdb-0.7.0.tar.gz (219 kB) Preparing metadata (setup.py):started Preparing metadata (setup.py): finished with status 'done'Requirement already satisfied: aiosignal>=1.1.2 in/usr/local/lib/python3.10/site-packages (fromaiohttp=3.5.2->faust[rocksdb]) (1.2.0) Requirement alreadysatisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/site-packages(from aiohttp=3.5.2->faust[rocksdb]) (21.2.0) Requirementalready satisfied: frozenlist>=1.1.1 in/usr/local/lib/python3.10/site-packages (fromaiohttp=3.5.2->faust[rocksdb]) (1.2.0) Requirement alreadysatisfied: charset-normalizer=2.0 in/usr/local/lib/python3.10/site-packages (fromaiohttp=3.5.2->faust[rocksdb]) (2.0.9) Requirement alreadysatisfied: multidict=4.5 in/usr/local/lib/python3.10/site-packages (fromaiohttp=3.5.2->faust[rocksdb]) (5.2.0) Requirement alreadysatisfied: async-timeout=4.0.0a3 in/usr/local/lib/python3.10/site-packages (fromaiohttp=3.5.2->faust[rocksdb]) (4.0.2) Requirement alreadysatisfied: python-dateutil in /usr/local/lib/python3.10/site-packages(from croniter>=0.3.16->faust[rocksdb]) (2.8.2) Requirement alreadysatisfied: colorlog>=2.9.0 in /usr/local/lib/python3.10/site-packages(from mode=4.3.2->faust[rocksdb]) (6.6.0) Requirement alreadysatisfied: setuptools>=25 in /usr/local/lib/python3.10/site-packages(from python-rocksdb>=0.6.7->faust[rocksdb]) (57.5.0) Requirementalready satisfied: kafka-python=1.4.6 in/usr/local/lib/python3.10/site-packages (fromrobinhood-aiokafka=1.1.6->faust[rocksdb]) (1.4.7) Requirementalready satisfied: idna>=2.0 in/usr/local/lib/python3.10/site-packages (fromyarl=1.0->faust[rocksdb]) (3.3) Requirement already satisfied:six>=1.5 in /usr/local/lib/python3.10/site-packages (frompython-dateutil->croniter>=0.3.16->faust[rocksdb]) (1.16.0)
Buildingwheels for collected packages: python-rocksdb Building wheel forpython-rocksdb (setup.py): started ERROR: Command errored out withexit status 1: command: /usr/local/bin/python -u -c 'import io, os,sys, setuptools, tokenize; sys.argv[0] ='"'"'/tmp/pip-install-b8y7g4hs/python-rocksdb_b1c08993fd134ac4bc59e6f5d18bcd91/setup.py'"'"';file='"'"'/tmp/pip-install-b8y7g4hs/python-rocksdb_b1c08993fd134ac4bc59e6f5d18bcd91/setup.py'"'"';f= getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools importsetup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"','"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'bdist_wheel -d /tmp/pip-wheel-9_o4ek6zcwd: /tmp/pip-install-b8y7g4hs/python-rocksdb_b1c08993fd134ac4bc59e6f5d18bcd91/Complete output (64 lines): running bdist_wheel running buildrunning build_py creating build creatingbuild/lib.linux-x86_64-3.10 creatingbuild/lib.linux-x86_64-3.10/rocksdb copying rocksdb/interfaces.py ->build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/errors.py ->build/lib.linux-x86_64-3.10/rocksdb copyingrocksdb/merge_operators.py -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/init.py -> build/lib.linux-x86_64-3.10/rocksdbcreating build/lib.linux-x86_64-3.10/rocksdb/tests copyingrocksdb/tests/test_memtable.py ->build/lib.linux-x86_64-3.10/rocksdb/tests copyingrocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.10/rocksdb/testscopying rocksdb/tests/init.py ->build/lib.linux-x86_64-3.10/rocksdb/tests copyingrocksdb/tests/test_options.py ->build/lib.linux-x86_64-3.10/rocksdb/tests running egg_info writingpython_rocksdb.egg-info/PKG-INFO writing dependency_links topython_rocksdb.egg-info/dependency_links.txt writing requirements topython_rocksdb.egg-info/requires.txt writing top-level names topython_rocksdb.egg-info/top_level.txt reading manifest file'python_rocksdb.egg-info/SOURCES.txt' reading manifest template'MANIFEST.in' writing manifest file'python_rocksdb.egg-info/SOURCES.txt' copying rocksdb/_rocksdb.cpp-> build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/rocksdb.pyx -> build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/cache.pxd ->build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/comparator.pxd-> build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/env.pxd ->build/lib.linux-x86_64-3.10/rocksdb copyingrocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/merge_operator.pxd ->build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/options.pxd ->build/lib.linux-x86_64-3.10/rocksdb copying rocksdb/slice.pxd ->build/lib.linux-x86_64-3.10/rocksdb copyingrocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/status.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.10/rocksdbcopying rocksdb/table_factory.pxd ->build/lib.linux-x86_64-3.10/rocksdb copyingrocksdb/universal_compaction.pxd ->build/lib.linux-x86_64-3.10/rocksdb creatingbuild/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/comparator_wrapper.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/filter_policy_wrapper.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/memtable_factories.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/merge_operator_wrapper.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/slice_transform_wrapper.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp copyingrocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.10/rocksdb/cppcopying rocksdb/cpp/write_batch_iter_helper.hpp ->build/lib.linux-x86_64-3.10/rocksdb/cpp running build_extcythoning rocksdb/_rocksdb.pyx to rocksdb/_rocksdb.cpp/tmp/pip-install-b8y7g4hs/python-rocksdb_b1c08993fd134ac4bc59e6f5d18bcd91/.eggs/Cython-0.29.26-py3.10-linux-x86_64.egg/Cython/Compiler/Main.py:369:FutureWarning: Cython directive 'language_level' not set, using 2 fornow (Py2). This will change in a later release! File:/tmp/pip-install-b8y7g4hs/python-rocksdb_b1c08993fd134ac4bc59e6f5d18bcd91/rocksdb/_rocksdb.pyxtree = Parsing.p_module(s, pxd, full_module_name) building 'rocksdb._rocksdb' extension creating build/temp.linux-x86_64-3.10creating build/temp.linux-x86_64-3.10/rocksdb gcc -pthread-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.10 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.10/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall-Wextra -Wconversion -fno-strict-aliasing -fno-rtti rocksdb/_rocksdb.cpp:705:10: fatal error: rocksdb/slice.h: No suchfile or directory705 #include "rocksdb/slice.h" ^ compilation terminated. error: command '/usr/bin/gcc' failed with exit code 1---------------------------------------- Building wheel for python-rocksdb (setup.py): finished with status 'error' ERROR:Failed building wheel for python-rocksdb
Accordingly, you'll need to build and install RocksDB. This is separate from the installation of faust[rocksdb] with pip. That simply installs python-rocksdb, the Python interface to the underlying libraries.
In the docker directory, there are a similar set of files to the uberjar examples above, as well as a Dockerfile and a build-docker.sh script. As with building a JAR, we can add rocksdb as the KV store - first, by adding a dependency on com.xtdb/xtdb-rocksdb to deps.edn Default index and event-log directories are already configured within the sample xtdb.edn (allowing you to map the Docker directories and give the images persistent storage), we then need to add xtdb.kv.rocksdb/kv-store to the three modules:
To see how that works can be done by creating a test YugabyteDB cluster which allows you access to the operating system layer, such as using the Vagrant boxes, Docker containers, Kubernetes, using precompiled executables for Linux or MacOS or build it from source, and run a YugabyteDB database with any replication factor you like. 2ff7e9595c
Comments