Skip to content

Commit

Permalink
add how to use the ingest-attachment plugin
Browse files Browse the repository at this point in the history
  • Loading branch information
jensens committed Nov 13, 2023
1 parent 49811e1 commit 165ae07
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 5 deletions.
19 changes: 16 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,21 +135,34 @@ Example configuration files are provided in the ``/examples`` directory.
OpenSearch with Docker Compose
------------------------------

A docker-compose file ``docker-compose.yml`` to start an OpenSearch server is provided.
A docker-compose file ``docker-compose.yml`` and a ``Dockerfile`` to start an OpenSearch server is provided.

Precondition:

- Docker and docker-compose are installed.
- Max virtual memory map needs increase to run this: `sudo sysctl -w vm.max_map_count=262144` (not permanent, `see StackOverflow post <https://stackoverflow.com/questions/66444027/max-virtual-memory-areas-vm-max-map-count-65530-is-too-low-increase-to-at-lea>`_).

Enter the directory ``examples`` and start the server with ``docker-compose up``.
Steps to start the example OpenSearch Server with the ``ingest-attachment`` plugin installed:

- enter the directory ``cd examples``
- build the docker image with

```bash
docker buildx use default
docker buildx build --tag opensearch-ingest-attachment:latest Dockerfile
```
- start the server with ``docker-compose up``.

Now you have an OpenSearch server running on ``http://localhost:9200`` and an OpenSearch Dashboard running on ``http://localhost:5601`` (user/pass: admin/admin).
The OpenSearch server has the ``ingest-attachment`` plugin installed.
The plugin enables OpenSearch to extract text from binary files like PDFs.

Open another terminal.

An `.env` file is provided with the environment variables ready to use with the docker-compose file.
Run ``source examples/.env`` to load the environment variables.

Then start the celery worker with ``celery -A collective.elastic.ingest.celery.app worker -l debug``.

In another terminal window `run a Plone backend <https://6.docs.plone.org/install/index.html>`_ at ``http://localhost:8080/Plone`` with the add-on `collective.elastic.plone` installed.
There, create an item or modify an existing one.
You should see the indexing task in the celery worker terminal window.
Expand Down
2 changes: 2 additions & 0 deletions examples/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
FROM opensearchproject/opensearch:latest
RUN /usr/share/opensearch/bin/opensearch-plugin install --batch ingest-attachment
4 changes: 2 additions & 2 deletions examples/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
version: '3'
services:
opensearch-node1: # This is also the hostname of the container within the Docker network (i.e. https://opensearch-node1/)
image: opensearchproject/opensearch:latest # Specifying the latest available image - modify if you want a specific version
image: opensearch-ingest-attachment:latest # Specifying the freshly build image
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # Name the cluster
Expand All @@ -27,7 +27,7 @@ services:
networks:
- opensearch-net # All of the containers will join the same Docker bridge network
opensearch-node2:
image: opensearchproject/opensearch:latest # This should be the same image used for opensearch-node1 to avoid issues
image: opensearch-ingest-attachment:latest # Specifying the freshly build image
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
Expand Down

0 comments on commit 165ae07

Please sign in to comment.