Skip to content

Commit

Permalink
more precise vignette wording
Browse files Browse the repository at this point in the history
  • Loading branch information
shikokuchuo committed Sep 16, 2024
1 parent bd534b8 commit 1c83562
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 60 deletions.
112 changes: 57 additions & 55 deletions vignettes/mirai.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,15 @@ To wait for and collect the return value, use the mirai's `[]` method:

``` r
m[]
#> [1] 5.711646 4.885233 2.739455 4.676828 3.793265
#> [1] 4.117674 2.635838 4.644908 3.868147 2.815506
```
As a mirai represents an async operation, it is never necessary to wait for it. Other code can continue to be run. Once it completes, the return value automatically becomes available at `$data`.

``` r
m
#> < mirai [$data] >
m$data
#> [1] 5.711646 4.885233 2.739455 4.676828 3.793265
#> [1] 4.117674 2.635838 4.644908 3.868147 2.815506
```
For easy programmatic use of `mirai()`, '.expr' accepts a pre-constructed language object, and also a list of named arguments passed via '.args'. So, the following would be equivalent to the above:

Expand All @@ -82,7 +82,7 @@ args <- list(time = x$time, mean = x$mean)

m <- mirai(.expr = expr, .args = args)
m[]
#> [1] 3.258056 4.590492 4.025177 4.083115 4.346042
#> [1] 2.992972 5.722461 2.917491 3.053203 4.436955
```

[&laquo; Back to ToC](#table-of-contents)
Expand Down Expand Up @@ -163,8 +163,8 @@ for (i in 1:10) {
#> iteration 5 successful
#> iteration 6 successful
#> iteration 7 successful
#> iteration 8 successful
#> Error: random error
#> iteration 8 successful
#> iteration 9 successful
#> iteration 10 successful
```
Expand Down Expand Up @@ -200,12 +200,12 @@ status()
#>
#> $daemons
#> i online instance assigned complete
#> abstract://f40d2d86792ca2baca4732cc 1 1 1 0 0
#> abstract://d9d4b983bed55b336a6aaf19 2 1 1 0 0
#> abstract://a5a05c506bef860d74380df7 3 1 1 0 0
#> abstract://205cf5f5fdbe497650e1802a 4 1 1 0 0
#> abstract://06159b3b3ced66c1888b2a2c 5 1 1 0 0
#> abstract://46040c893387d18a98e5d42f 6 1 1 0 0
#> abstract://e8fc9ebfab5cc37527d9bcb5 1 1 1 0 0
#> abstract://ef183df51fce19b25926326d 2 1 1 0 0
#> abstract://cf3bf07799f7d04c8c1e5625 3 1 1 0 0
#> abstract://86a00a3f2458179f2bf3e2e7 4 1 1 0 0
#> abstract://99f979d222e678cf025229c8 5 1 1 0 0
#> abstract://31f921ec2f6a6a15a29d4f4e 6 1 1 0 0
```
The default `dispatcher = TRUE` creates a `dispatcher()` background process that connects to individual daemon processes on the local machine. This ensures that tasks are dispatched efficiently on a first-in first-out (FIFO) basis to daemons for processing. Tasks are queued at the dispatcher and sent to a daemon as soon as it can accept the task for immediate execution.

Expand Down Expand Up @@ -237,7 +237,7 @@ status()
#> [1] 6
#>
#> $daemons
#> [1] "abstract://1d55e5f863d78c92c93aa11e"
#> [1] "abstract://673a3eeb52a467132770f634"
```
This implementation sends tasks immediately, and ensures that tasks are evenly-distributed amongst daemons. This means that optimal scheduling is not guaranteed as the duration of tasks cannot be known *a priori*. As an example, tasks could be queued at a daemon behind a long-running task, whilst other daemons are idle having already completed their tasks.

Expand Down Expand Up @@ -265,11 +265,11 @@ m <- mirai(capture.output(str(con)))
m[]
#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots"
#> [2] " ..@ ptr :<externalptr> "
#> [3] " ..@ dbname : chr \"/tmp/Rtmplaxmyl/file13b255ceefb79\""
#> [3] " ..@ dbname : chr \"/tmp/RtmpCBzAfq/file1755316a58817\""
#> [4] " ..@ loadable.extensions: logi TRUE"
#> [5] " ..@ flags : int 70"
#> [6] " ..@ vfs : chr \"\""
#> [7] " ..@ ref :<environment: 0x5a0511e7e9a8> "
#> [7] " ..@ ref :<environment: 0x5d7760539298> "
#> [8] " ..@ bigint : chr \"integer64\""
#> [9] " ..@ extended_types : logi FALSE"
```
Expand Down Expand Up @@ -370,7 +370,7 @@ status()
#> [1] 0
#>
#> $daemons
#> [1] "tcp://hostname:40691"
#> [1] "tcp://hostname:33355"
```
The number of daemons connecting to the host URL is not limited and network resources may be added or removed at any time, with tasks automatically distributed to all connected daemons.

Expand Down Expand Up @@ -461,10 +461,10 @@ daemons(n = 2, url = host_url())
#> [1] 2
launch_remote(1:2)
#> [1]
#> Rscript -e 'mirai::daemon("tcp://hostname:33945",rs=c(10407,1532471919,1128930788,-1010662507,654765906,-1979265365,-1365430800))'
#> Rscript -e 'mirai::daemon("tcp://hostname:42071",rs=c(10407,-2097114620,-2132598475,-540653454,-1947742901,-1688783856,109674705))'
#>
#> [2]
#> Rscript -e 'mirai::daemon("tcp://hostname:32963",rs=c(10407,-1278619456,1970930841,1676950159,1779697564,-1289813301,1754512148))'
#> Rscript -e 'mirai::daemon("tcp://hostname:35173",rs=c(10407,1934912827,1041955822,902374120,-191378234,979262844,-895413437))'
daemons(0)
#> [1] 0
```
Expand All @@ -491,37 +491,37 @@ The generated self-signed certificate is available via `launch_remote()`. This f
``` r
launch_remote(1)
#> [1]
#> Rscript -e 'mirai::daemon("wss://hostname:43649/1",tls=c("-----BEGIN CERTIFICATE-----
#> Rscript -e 'mirai::daemon("wss://hostname:35557/1",tls=c("-----BEGIN CERTIFICATE-----
#> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h
#> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw
#> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV
#> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP
#> ADCCAgoCggIBAM3Y3VeqOu5z8cdFwQUANT/I6DD+cE9hsHbloA8rMTRFxgZRJRYX
#> QlIaOrpF7hxz2Y/b72DGjUZmbeF5hToXD2HL2GDUyQ7iWE7lJP9ksjRy4AAjpA4r
#> elkS2EUT4NBZTAapgAHv/WkT4u+KjCcD6WNaPulaLOpOCIARl7NGjapvbgnz5KuR
#> wWdRQoUzgtDj3aVLKWFbcGgwKhIDrpq9mkObAjvTP+EAcDUM3zTRFYEjX7k4SCrJ
#> KXABxex7zlCIrqTkWQFHZkg+ozlvHh6vK+p3H1o+lkxzeLRB7Dg4q17py+IOdw4I
#> sR3S00N9m4VU1CcKx2PpsmbmkTQ9dJlzLa8LjQOfCWQ01BQoDfetivq1iy36Pl1k
#> +R2Jix67jpBvptSZcOV3m0yhNiismwsee9HclrbpcInjSblYNFhgIwiokVj2BRgF
#> /4HmTfFpvu8fsquasxqfyAHrcZOT1N6c6bjWf72GiHDh0sh9furE1EhN8Uun40IG
#> ENwdUh8CmY1Wp62PdZpJTRsp71IefZ8WVlSvgEOmOUc5ksnsST8csEzzEQwWVfjl
#> Auk0nPP4yA0rLJ7kZC6XD/4WiPSwkSuXgAHwzXbalA3XdKVeu+goIp9XpvOwZhPX
#> 6uJdZ3b8blnDNXAEekZrk++mF+f8xG0LY4mi8VPG0gcXIwBJp98P5gzJAgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFEZByPcQfoAOeLuNQ9BM
#> HjDFIj6JMB8GA1UdIwQYMBaAFEZByPcQfoAOeLuNQ9BMHjDFIj6JMA0GCSqGSIb3
#> DQEBCwUAA4ICAQBoEyaVyK2VuvDHsqYaGGTCiBV02nusa2IySUTipcnB0+5oX89t
#> 1hhngxZiCnnXAdPLqyVFqinHwNiKK7NwQRrMIRa2zDGYWo9Q1Jd06Q1DeSDD+uxA
#> AwV1UUI4wxcms99EcQN5/oCzprfenW89ipR7bVev7e/yaZQlAMYoAU5Y2cgJHky+
#> rmYOImk2gG6XTuCsmF65MFZ+EkuTGaMb0PRz4/81wys13DD8m/kBVBsT8wIOmQRf
#> fHTImRxUVLsvkGLYlmM9y2YL/xPl5QSapcn4/1FbZyX3SyfGjHCINxV3FkKWo3ka
#> VP5bDwD/vGMFPicOKCiSqWx+KjNz/Lchk8SyACY62F2m5T5ztdT/Jfke26jgP2Bt
#> 9SiUQs+2wUFrt6WTKNbQrnR2BYUYslO15HVEfr9rMhCgShKISJfFOYjnHWuPZ8SF
#> FuBBW63cliGt7nfT4LOVwQtf6Pea8Uy7xTSJG8JQNFDgNCtsB6HCpHIwpnxcArRy
#> T8rfGcIqQk4OiQlMSHpvA6FSt/5qXez4ynqUvUeuaE1WxLnlMJMAhmQ06V/eipNM
#> g0wYeDRlERMEaAnjQ02/Hnx7zhd7dAPY4zw3G7yPPKdspJMLAArrr4IDwuJm4yVH
#> 8j1+RVDO88KbdfROgLkp6zyZnt51lvGYnJFfoINzdNLxKYa2rZ/KqSBnzQ==
#> ADCCAgoCggIBAJ2crLEfixQUdm3nGAZl/4tktF1mL0/Egdl/IkgQ5PjfkfvSc9Dl
#> tCKx3ODSu9wCxUFa1IBmrwEfENFOtiOw9wqj3CYLYyGxeMY+WJsaJbgg1n7YLWJA
#> Bk+NTF73rGoMiEmtGXDj6Z4yMXJyqgu8tAuk2lMdQOrhsUYctHPknTEJCZltw36d
#> waAxuJWsSwINXhHC6VxyNJAkAUO8I7r4OTb3Et9nu9tHYKs1Z1lTTrgsD66uinjX
#> 2/xneZwYLvdtlQO/oNjHZzzBgdPJrfVFDSp7I32Tbb8RVz5ED0DS719GKbkRzpul
#> bOCoo4zedCfoVD7BmR3WouxOcx9QRhPBDlIWlbhasdIeBq5NWxtftfuQ3LJ0cV2D
#> 782QomjGLSkSTKZPt7sfHd82Uhz44H9CPIJPI1buDtDZvd81JwUluPY3IyAJeroq
#> Fx1268h7MQLUl7p2f+hR44/hO4hr815e7Zu8Xv7U9igWV6sRB8r5MLwB/JNM6mfm
#> ZEuOTHhot9EUWF1iMpg5u0eZln02ph1MzPJDA1f2qfl4wTE7bzsqy3Us1k45RDV1
#> G8kwHWvYwUaJydqsh2uUzM/uaH2XmEJ8HRlkA2Kv7EnutrdKlPotmnWA/C085Lr1
#> NC5NSmPOLz4Duwpv9dB86kn7GFqX1IdmBcOMB0hJirx2vUCbP3vMC2CZAgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFIdA0zFxm2n3rZpn6rKd
#> mElSOwd3MB8GA1UdIwQYMBaAFIdA0zFxm2n3rZpn6rKdmElSOwd3MA0GCSqGSIb3
#> DQEBCwUAA4ICAQAagc2W2aKw42+wNUsqz/Kq5ezWNMQJ6vayfsCp1Cr52mUOaXkB
#> gyddeLQjAy7XEHgqan+akB7aloQYKqD2q2EBhWgUAhL+BjeIZ5F6kTTgOhCaHjXm
#> wYZlsCP2wLd/XQjELC93qRTUeHwlSgc5g612BDIqrAGjKKhrnM3LdWmvMtFDhZYr
#> 3I38toTw/qlJMKYNF8WrESPlF4reLPE7KlC4z09HvgvlUsMIp0sRRh6Ob05OD/NY
#> 1IhvQbBBL0gEw4JrZQ9b7j0l40oebJuOLetL4FjqXfcFRFaSaL4kPBgtZdTd/fge
#> mQnYmAfGY7384Q0OxgQu3V6qsFrKCO3M0H9w5jdIUxMmPlABHirLj8wEI5T+2Q2n
#> 7awZoYw8kpGYDMiB3YcWBf1yUNBcFAw4qJdlJaniTbSrRou+jIH7a+0W+fxudcxo
#> rRrfX9fC4YPYbcwhh7+LqlYMVkMLFND1yUGOyDm+n6mbIYl1uFCary3/qNJKyNwM
#> 53wMcHYglF1/82Bgu5esag4SAy47Wkd9oRCHYN3RJ9dYF76ZIZ6pJVUB4n3tic/s
#> ypls+wGUUTmqsfrJFXSXyTcttDEhGumjhS62EWSV9Rk3u+XTc3IwZtLlTSSQ2aj1
#> LNXd5d6cXxBTqNBbNO+4UUTFLZTyrNLgyTL4JJu1p8/B9RZG97KeSIH2Tg==
#> -----END CERTIFICATE-----
#> ",""),rs=c(10407,2099570300,-54731443,-674816086,112852259,-772965368,-1501513111))'
#> ",""),rs=c(10407,908205630,857300391,-1833403972,1436813197,-263637782,1463370083))'
```
The printed value may be deployed directly on a remote machine.

Expand Down Expand Up @@ -753,10 +753,10 @@ daemons(4)
vec <- c(1, 1, 4, 4, 1, 1, 1, 1)
system.time(mirai_map(vec, Sys.sleep)[])
#> user system elapsed
#> 0.005 0.002 4.006
#> 0.004 0.001 4.006
system.time(parLapply(cl, vec, Sys.sleep))
#> user system elapsed
#> 0.006 0.006 8.009
#> 0.008 0.002 8.010
```
`.args` is used to specify further constant arguments to `.f` - the 'mean' and 'sd' in the example below:

Expand All @@ -766,13 +766,13 @@ with(
mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[]
)
#> [[1]]
#> [1] 19.47468
#> [1] 18.82132
#>
#> [[2]]
#> [1] 21.37112 20.30177
#> [1] 20.00699 18.41221
#>
#> [[3]]
#> [1] 17.93560 19.61252 16.09829
#> [1] 23.36931 16.06004 18.91166
```
Use `...` to further specify objects referenced but not defined in `.f` - the 'do' in the anonymous function below:

Expand All @@ -787,13 +787,13 @@ ml
#> < mirai map [3/3] >
ml[]
#> $a
#> [1] "4c"
#> [1] "ea"
#>
#> $b
#> [1] 06 48
#> [1] cf e7
#>
#> $c
#> [1] "b38df3"
#> [1] "f3a146"
```
Use of `mirai_map()` assumes that `daemons()` have previously been set. If not then one (non-dispatcher) daemon is set to allow the function to proceed. This ensures safe behaviour, but is unlikely to be optimal, so please ensure daemons are set beforehand.

Expand Down Expand Up @@ -829,19 +829,21 @@ daemons(0)

mirai as a framework is designed to support completely transparent and inter-operable use within packages. A core design precept of not relying on global options or environment variables minimises the likelihood of conflict between use by different packages.

There are hence not many requirements of package authors. However, the following notes may be helpful:
There are hence not many requirements of package authors.

The following notes may be helpful nevertheless:

- `daemons()` settings should usually be left to end-users. Either provide guidance on how to do this or point them to `mirai` documentation. If however, your package wishes to set default settings for instance, `daemons()` should always be called specifying `force = FALSE`. This ensures that any prior user setting is respected, and that daemons set elsewhere are not prematurely terminated.
- `daemons()` settings should usually be left to end-users. Users may be pointed to mirai documentation if desired. If however, your package wishes to set default settings for instance, `daemons()` should always be called specifying `force = FALSE`. This ensures that any prior user settings are respected, and that daemons set elsewhere are not prematurely terminated.

- Consider exposing the '.compute' argument in package functions, mapping to the '.compute' argument of `mirai()`, so that the end-user may choose where to run your functions if they have multiple compute profiles set up.
- Consider exposing a `.compute = "default"` argument in package functions, mapping to the '.compute' argument of `mirai()`, so that the end-user may choose where to run your functions if they have multiple compute profiles set up.

- Calling package functions in a mirai requires namespacing the call, or an equivalent way to access the functions, i.e.
```r
mirai(mypkg::my_func())
```

- Do not rely on the shape or contents of the `status()` daemons matrix, as this user interface is subject to change at any time. There is a developer interface `nextget()`, for querying documented values such as 'urls'. Note: only these specifically-documented values are supported interfaces.
- Do not rely on the shape or contents of the `status()` daemons matrix, as this user interface is subject to change at any time. There is a developer interface `nextget()`, for querying values such as 'urls' described in the function documentation. Note: only these specifically-documented values are supported interfaces.

- Testing on CRAN should respect it's 2-core usage limit. This practically means limiting tests to using one daemon (with `dispatcher = FALSE`) to ensure that only one additional process is used. Always reset daemons at the end and then allow at least a one-second sleep to ensure all background processes have exited properly. There are of course no limits for testing outside of CRAN, and a full suite of tests using more daemons options may be run locally.
- Testing on CRAN should respect it's 2-core usage limit. This practically means limiting tests to using one daemon (with `dispatcher = FALSE`) to ensure that only one additional process is used. Always reset daemons when done and then allow at least a one-second sleep to ensure all background processes have properly exited. These limits only apply to CRAN of course, and a full suite of tests involving further daemons options may be run in other contexts.

[&laquo; Back to ToC](#table-of-contents)
12 changes: 7 additions & 5 deletions vignettes/mirai.Rmd.orig
Original file line number Diff line number Diff line change
Expand Up @@ -617,19 +617,21 @@ daemons(0)

mirai as a framework is designed to support completely transparent and inter-operable use within packages. A core design precept of not relying on global options or environment variables minimises the likelihood of conflict between use by different packages.

There are hence not many requirements of package authors. However, the following notes may be helpful:
There are hence not many requirements of package authors.

- `daemons()` settings should usually be left to end-users. Either provide guidance on how to do this or point them to `mirai` documentation. If however, your package wishes to set default settings for instance, `daemons()` should always be called specifying `force = FALSE`. This ensures that any prior user setting is respected, and that daemons set elsewhere are not prematurely terminated.
The following notes may be helpful nevertheless:

- Consider exposing the '.compute' argument in package functions, mapping to the '.compute' argument of `mirai()`, so that the end-user may choose where to run your functions if they have multiple compute profiles set up.
- `daemons()` settings should usually be left to end-users. Users may be pointed to mirai documentation if desired. If however, your package wishes to set default settings for instance, `daemons()` should always be called specifying `force = FALSE`. This ensures that any prior user settings are respected, and that daemons set elsewhere are not prematurely terminated.

- Consider exposing a `.compute = "default"` argument in package functions, mapping to the '.compute' argument of `mirai()`, so that the end-user may choose where to run your functions if they have multiple compute profiles set up.

- Calling package functions in a mirai requires namespacing the call, or an equivalent way to access the functions, i.e.
```r
mirai(mypkg::my_func())
```

- Do not rely on the shape or contents of the `status()` daemons matrix, as this user interface is subject to change at any time. There is a developer interface `nextget()`, for querying documented values such as 'urls'. Note: only these specifically-documented values are supported interfaces.
- Do not rely on the shape or contents of the `status()` daemons matrix, as this user interface is subject to change at any time. There is a developer interface `nextget()`, for querying values such as 'urls' described in the function documentation. Note: only these specifically-documented values are supported interfaces.

- Testing on CRAN should respect it's 2-core usage limit. This practically means limiting tests to using one daemon (with `dispatcher = FALSE`) to ensure that only one additional process is used. Always reset daemons at the end and then allow at least a one-second sleep to ensure all background processes have exited properly. There are of course no limits for testing outside of CRAN, and a full suite of tests using more daemons options may be run locally.
- Testing on CRAN should respect it's 2-core usage limit. This practically means limiting tests to using one daemon (with `dispatcher = FALSE`) to ensure that only one additional process is used. Always reset daemons when done and then allow at least a one-second sleep to ensure all background processes have properly exited. These limits only apply to CRAN of course, and a full suite of tests involving further daemons options may be run in other contexts.

[&laquo; Back to ToC](#table-of-contents)

0 comments on commit 1c83562

Please sign in to comment.