-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fixes #22 Build Project on Azure Pipelines
- Loading branch information
1 parent
bde782a
commit 2afec13
Showing
2 changed files
with
55 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Indexer | ||
|
||
[![Build Status](https://dev.azure.com/branham-player/indexer/_apis/build/status/branham-player.indexer%20(1)?branchName=develop)](https://dev.azure.com/branham-player/indexer/_build/latest?definitionId=3&branchName=develop) | ||
|
||
A parser which indexes unstructured collections of data representing William Branham's complete sermon library and structures them for loading into a data ingester. | ||
|
||
This project is part of a three-part system which collectively stores, indexes, and then outputs a collection of sermons as JSON files: | ||
|
||
1. [Original Sources](https://github.com/branham-player/original-sources): Sermon metadata from various online data sources | ||
1. **[Indexer](https://github.com/branham-player/indexer): This project, takes the information from the above source and processes it as output into the next source** | ||
1. [Golden Dataset](https://github.com/branham-player/golden-dataset): JSON files which are generated by the indexer and manually uploaded to the repository | ||
|
||
## Run this Project | ||
|
||
To run this project and generate the intended output, you will need the latest version of [NodeJS](https://nodejs.org/en/download/). Once that is installed, run these commands: | ||
|
||
```bash | ||
git clone https://github.com/branham-player/indexer.git | ||
git clone https://github.com/branham-player/original-sources.git | ||
|
||
cd indexer | ||
npm install | ||
npm run all | ||
``` | ||
|
||
The result of a successful execution is the presence of three new files in the root folder: | ||
|
||
- `full.json`: A complete dataset which contains all of the information the program could gather from the original sources | ||
- `condensed.json`: A shortened version of the `full.json` file which contains the most essential pieces of information for the everyday user | ||
- `months.json`: Counts the number of years and months which are present in `full.json` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
name: $(SourceBranchName) $(Rev:r) | ||
pr: none | ||
|
||
pool: | ||
vmImage: 'windows-2019' | ||
|
||
steps: | ||
- script: | | ||
npm install | ||
displayName: 'Install package.json Dependencies' | ||
|
||
- script: | | ||
git clone -q https://github.com/branham-player/original-sources.git | ||
displayName: 'Pull Original Sources Repository' | ||
|
||
- script: | | ||
md output | ||
node --experimental-modules database.mjs ./original-sources/ ./output/full.json ./output/condensed.json | ||
node --experimental-modules months.mjs ./output/full.json ./output/months.json | ||
displayName: 'Build the Database' | ||
|
||
- task: PublishPipelineArtifact@0 | ||
displayName: 'Save JSON Files' | ||
inputs: | ||
targetPath: 'output' |