Releases: Pometry/Raphtory
Releases · Pometry/Raphtory
v0.11.3
Parallel python loaders
Through some elegant dancing around locks, the pandas and parquet loaders now ingest into Raphtory’s underlying graph shards with minimal contention between threads. This has led to an order of magnitude improvement in ingestion speed in several of our use cases!
An example of this can be seen below where the 129 million edges of the Graph500 SF23 dataset are ingested in 25 seconds on a laptop!
Other minor bug fixes
- Don't drop the update in the Cache if the write fails by @ljeub-pometry in #1745
- fix typo in cdf and ccdf functions by @narnolddd in #1748
- More cache writer error handling improvements by @ljeub-pometry in #1747
- fix latest_time for exploded edge on PersistentGraph for edges that are deleted at the same time as created by @ljeub-pometry in #1752
- implementation of temporal rich club by @narnolddd in #1692
- Write-locked graph storage implementation for faster bulk loaders by @ljeub-pometry in #1741
- Release v0.11.3 by @github-actions in #1753
Full Changelog: v0.11.2...v0.11.3
v0.11.2
Release v0.11.2
- Publish to crates.io
- Publish to PyPi
- Make Tag
- Release to Github
- Auto-generated by [create-pull-request] triggered by release action [1]
[1]: https://github.com/peter-evans/create-pull-request
- Auto-generated by [create-pull-request] triggered by release action [1]
v0.11.1
Release 0.11.1
Bug Fixes
- Exposed
delete
on edge in python - Fixed missing DTime parsing on Python properties
- Fixed a bug in import_node(s)/import_edge(s) when the graph is indexed with ints
Graphql
- Added new
RemoteGraph
,RemoteNode
andRemoteEdge
classes to wrap updates to graph on a server. These functions can be seen below - Made Properties optional in add_updates for Node and Edge in Graphql
- fixed a bug in batch add nodes, where you couldn't just add the node_type.
client = graphql.RaphtoryClient(url="http://localhost:1736")
### RaphtoryClient functions
client.new_graph(path)
### RemoteGraph Functions
client.graph(path).add_node()
client.graph(path).add_nodes()
client.graph(path).add_edge()
client.graph(path).add_edges()
client.graph(path).delete_edge()
client.graph(path).add_properties()
client.graph(path).add_constant_properties()
client.graph(path).update_constant_properties()
### RemoteNode functions
client.graph(path).node(name).add_constant_properties()
client.graph(path).node(name).set_node_type()
client.graph(path).node(name).update_constant_properties()
client.graph(path).node(name).add_updates()
### RemoteEdge functions
client.graph(path).edge(src,dst).delete()
client.graph(path).edge(src,dst).add_constant_properties()
client.graph(path).edge(src,dst).update_constant_properties()
client.graph(path).edge(src,dst).add_updates()
##Misc
- Tided up graphql pytests as having a single file was driving me mad
- Broke the graphql python module into smaller files
What's Changed
- Upgrade Datafusion to 52 and fix the schema errors by @fabianmurariu in #1716
- Graphql helper by @miratepuffin in #1733
- Release v0.11.1 by @github-actions in #1739
Full Changelog: v0.11.0...v0.11.1
v0.11.0
Release v0.11.0
Cached Graph
- We have updated the on-disk format of the graph to now be stable across versions.
- This new storage format is also updatable, allowing deltas to be inserted into an already existing file instead of having to resave the whole graph.
- You can now 'cache' a graph, attaching a file to it, and periodically save all changes which have been inserted. Great for checkpointing! See the example below:
from raphtory import Graph
g= Graph() #Create new graph
g.add_edge(1,1,2)
g.add_edge(3,4,5)
g.cache("example_graph.raph") #add some updates and write these to a file - keeping the file attached to the graph
g.add_node(2,3)
g.add_edge(10,2,3)
g.write_updates() #Add some new updates and write these out
GraphQL
- The GraphQL server has had a massive overhaul, allowing it to now create, update, and manage a working directory full of graphs.
- Graphs will now persist across server runs based on the new cache-graph file format described above.
- These new APIs also allow you to export subgraphs from one graph into another.
A basic example of this can be seen below:
from raphtory import graphql
server = graphql.GraphServer("graphs/")
server.run()
client = server.get_client() #run a local server and get the client
#create a new graph in the 'example_graphs' namespace
client.query("""
mutation {
newGraph(path:"example_graphs/graph1",graphType:EVENT)
}
""")
#add an edge into the new graph
client.query("""
query add_edge{
updateGraph(path: "example_graphs/graph1") {
addEdge(time:1,src:"node1",dst:"node2",properties: [
{key: "new", value: "new props are awesome"}
{key: "number", value: 0.7}
{key: "map", value: { a: "hi"}}
]){success}
}
}""")
Pandas and Parquet loader changes:
- We have placed time as the second parameter within the
graph.load_x_from_y
to matchg.add_node
andg.add_edge
. - All parameters named props in the
graph.load_x_from_y
functions have been updated toproperties
. - The
Layer_in_df
flag has been replaced bylayer
andlayer_col
which will now throw an error if both are set. - The
node_type_in_df
flag has been replaced bynode_type
andnode_type_col
which will now throw an error if both are set. - You can now set the node_type via
load_node_props_from_x
functions. - We have removed the static
load_from_pandas
andload_from_parquet
functions from the graph as these had WAY too many arguments and were all around very confusing. Instead you can now create a graph first and useload_edges_from_X
etc. - The Datareader behind the Pandas and Parquet loaders now chunks the data and loads these in in part, drastically reducing the memory overhead for large datasets on ingestion.
Minor changes
- Optimised the way node IDs are stored/referenced, greatly increasing both lookup and insertion speed. This does mean, however, that a graph can only have String XOR Integer ids now, not both at the same time.
- We have updated to rust 1.80.0
What's Changed
- Fix release by @fabianmurariu in #1693
- make the benchmark names consistent again by @ljeub-pometry in #1691
- Fix benchmark names by @ljeub-pometry in #1694
- Pre multi graph by @fabianmurariu in #1695
- Feature/more merge tests by @ljeub-pometry in #1702
- Features/cached graphs by @shivam-880 in #1656
- Feature/incremental protobuf by @ljeub-pometry in #1706
- update rust version for readthedocs by @ljeub-pometry in #1708
- Make sure the string interning happens on decoding protobuf files by @ljeub-pometry in #1711
- much cleaner and easier way to test all the features compile by @ljeub-pometry in #1710
- Post cache graphql fixes by @miratepuffin in #1709
- Fix/dfview by @shivam-880 in #1712
- Feature/cached graph by @ljeub-pometry in #1718
- Tidy/ingestor by @shivam-880 in #1714
- GraphQL mutation apis by @ljeub-pometry in #1719
- Update README.md by @miratepuffin in #1726
- Release v0.11.0 by @github-actions in #1727
Full Changelog: v0.10.0...v0.11.0
v0.10.0
What's Changed
- impl node_types for disk graph by @iamsmkr in #1641
- move input node and hashing code to raphtory-api by @ljeub-pometry in #1671
- Bump urllib3 from 2.2.1 to 2.2.2 in /docs in the pip group across 1 directory by @dependabot in #1663
- Parquet loader by @iamsmkr in #1666
- Improve python extensibility by @miratepuffin in #1677
- Make the edge storage more columnar to reduce memory usage by @fabianmurariu in #1670
- Stable serialise by @fabianmurariu in #1683
- Disk merge by @ljeub-pometry in #1678
- Add Python type stubs by @hallofstairs in #1682
- Cleanup by @ljeub-pometry in #1689
- Release v0.10.0 by @github-actions in #1690
New Contributors
- @hallofstairs made their first contribution in #1682
Full Changelog: v0.9.3...v0.10.0
v0.9.3
Release v0.9.3
- Publish to crates.io
- Publish to PyPi
- Make Tag
- Release to Github
- Auto-generated by [create-pull-request] triggered by release action [1]
[1]: https://github.com/peter-evans/create-pull-request
- Auto-generated by [create-pull-request] triggered by release action [1]
v0.9.2
New functionality and API changes
- PersistentGraphs are now available as properties on the graph #1596
- EventGraphs and PersistentGraphs can now be converted into eachother via
into_graph
andinto_persistent_graph
#1596 - Changed the return type of
layer_name
andtime
from Option to Result - this means when they are called incorrectly the user will get an error (explaining when these functions should be used) instead of a None, which could easily be confused for the edge not having a layer name or timestamp. #1593 - Added
unique
andordered_dedupe
functions on temporal properties #1603 - Added a Temporal Bipartite projection #1396
- Added node type filtering on
Nodes
PathFromGraph
andPathFromNode
#1510
GraphQL Improvements
- Added Microsoft SSO/Authentication #1629
- Set in_/out_/edges to return
GQLEdges
objects which can be correctly filtered and paged. #1585 - Set in_/out_/neighbours to return
GQLPathFromNode
objects which can be correctly filtered and paged. #1585 - Added an optional
keys
argument to the GQLPropertiesvalues
function - meaning you can select a subset of the properties on the nodes without having to callvalue
over and over. #1585 - Added Apollo event tracing to find bottlenecks in your queries #1585
- Removed node/edge filtering as it was ineffecient - all functionality is available now that graphql APIs are in parity with python/rust #1585
- Removed several old/ultra specific functions which can now be better served by the standard APIs #1585
Bug fixes and performance improvments
- Drastically improved the performance of the Eval APIs used for writing algorithms in Raphtory - this should see all algorithms written this way run faster. #1617 #1622
- Reworked the temporal motif algorithms to use more efficient internal functions and data structures #1623
- Fixed an issue with exploded edges panicking if a requested property wasn't present #1592
- Graphql can now correctly index Boolean properties #1597
- Exploded edges now return the correct value when calling
first
andlast
#1594 - Fixed an issue in saving/loading graphs in python where save used relative paths, but load used absolute.
- Added several new benchmarks to ensure no regressions in performance when making larger core changes.
- Several documentation fixes
Structural Changes
- There are several large structural refactors as part of this release, with another planned for 0.10.0 - these are to enable us to make both the Rust and Python libraries more extensible for developers building plugins. #1608 #1631
What's Changed
- Bump tqdm from 4.65.0 to 4.66.3 by @dependabot in #1588
- Bump jinja2 from 3.1.2 to 3.1.4 by @dependabot in #1589
- fix issue with missing properties for exploded edges causing panic #1592 by @iamsmkr in #1595
- Fix/graphql by @iamsmkr in #1597
- fix exploded edges latest time #1591 by @iamsmkr in #1594
- fixes #1583 by @iamsmkr in #1596
- Bump scrapy from 2.11.1 to 2.11.2 by @dependabot in #1601
- Graphql improvements by @miratepuffin in #1585
- Os master squashed by @fabianmurariu in #1608
- add hide_arrow_dep.sh by @fabianmurariu in #1612
- Fix readthedocs by @fabianmurariu in #1613
- Feature/layer name time by @iamsmkr in #1593
- Performance fixes for the eval apis by @ljeub-pometry in #1617
- Fix python non arrow by @fabianmurariu in #1616
- make local_clustering_coefficient benchmark actually useful again by @ljeub-pometry in #1618
- add raphtory-arrow to .gitignore to avoid accidental commits/push by @fabianmurariu in #1620
- Raphtory api by @fabianmurariu in #1621
- Fix/performance by @ljeub-pometry in #1622
- impl unique temporal props by @iamsmkr in #1603
- Temporal bipartite projection by @narnolddd in #1396
- Bump requests from 2.31.0 to 2.32.0 by @dependabot in #1607
- impl node type filter for nodes, pathfromgraph and pathfromnode by @iamsmkr in #1606
- run rust tests without arrow enabled to check that still works by @ljeub-pometry in #1626
- supported empty node types while node type filtering by @iamsmkr in #1624
- fix arrowgraph for gql by @iamsmkr in #1619
- fix issue with path while saving and loading graphs by @iamsmkr in #1627
- Motifs tidy ii by @narnolddd in #1623
- Refactor pre release by @fabianmurariu in #1631
- Added Microsoft Authentication to GraphQL by @Haaroon in #1629
- change key and perms for benchmark workflow by @Haaroon in #1633
- disbale internal dispatch by @Haaroon in #1634
- simplify the testing by @ljeub-pometry in #1632
- Update readme.md by @Haaroon in #1636
- Release v0.9.0 by @github-actions in #1637
- update pometry-storage to the right version of raphtory-api by @fabianmurariu in #1638
- Added dryrun flag as now required for release by @miratepuffin in #1639
- Fixed github release by @miratepuffin in #1640
- github actions are sometimes stupid by @fabianmurariu in #1642
- add metadata required for release by @fabianmurariu in #1643
- fix the zip dependency by @fabianmurariu in #1644
- Release v0.9.1 by @github-actions in #1645
- Remove par iter from hot path by @fabianmurariu in #1647
- Fixed documentation to state nodes instead of edges for find_nodes by @brandon-haugen in #1648
- Release v0.9.2 by @github-actions in #1652
New Contributors
- @brandon-haugen made their first contribution in #1648
Full Changelog: v0.8.1...v0.9.2
v0.8.1
New Features
- Exposed
update_constant_properties
on both the nodes and edges (was previously only available on the graph)
Version changes
- Removed support for python 3.7
- Bumped rust from 1.75 to 1.77
Bug fixes
- Fixed a bug in the property aggregation methods where None's were being included in the results.
- Changed the notebook argument in
to_pyvis
to befalse
by default.
v0.8.0
What's Changed
- Trying to fix the CI but idk by @Haaroon in #1430
- Bug/pypi publish ring error by @Haaroon in #1431
- Fix the semantics for edge history when using deletions by @ljeub-pometry in #1429
- Fixed Python Build Errors by @Haaroon in #1432
- Helper function for getting the first/last update in a entities history by @miratepuffin in #1434
- Add deletion history and is_valid check for edges by @ljeub-pometry in #1433
- frutchreingold algorithm and layout in raphtory and gql by @Haaroon in #1435
- Clean up dependencies and bump versions by @ljeub-pometry in #1438
- add version attribute to raphtory and fix the release in the docs by @ljeub-pometry in #1440
- Add rust version to readme by @narnolddd in #1441
- fix in out edges by @iamsmkr in #1442
- Added timezoned datetime support to python by @miratepuffin in #1443
- Bump fonttools from 4.42.1 to 4.43.0 in /docs by @dependabot in #1450
- Bump fonttools from 4.42.1 to 4.43.0 by @dependabot in #1449
- change the windowing semantics to not enforce nesting by default by @ljeub-pometry in #1445
- New implementation for reingold algorithm by @ricopinazo in #1451
- Add Nodes object to GQL to support window filtering by @ljeub-pometry in #1452
- fix is_valid on GraphWithDeletions by @ljeub-pometry in #1453
- Edges api improvements by @ljeub-pometry in #1455
- Python plugins for document search by @ricopinazo in #1454
- add support for edges object to GraphQl by @ljeub-pometry in #1456
- Make sure constant edge properties can't be added to layers that don't have updates for the edge by @ljeub-pometry in #1457
- Add graph documents and storage for vectorised graphs by @ricopinazo in #1439
- Clean up and improve layers api by @ljeub-pometry in #1459
- Fix string id edge cases by @ljeub-pometry in #1460
- Allow saving arbitrary serialized data to nodes by @louisch in #1461
- Bump pillow from 10.0.1 to 10.2.0 in /docs by @dependabot in #1464
- Bump pillow from 10.0.1 to 10.2.0 by @dependabot in #1465
- Added to_node_df and to_edge_df as core functions into graph.rs in py by @Haaroon in #1463
- Move to graph_view to support windows+layers by @Haaroon in #1467
- Shared Neighbours function by @rachchan in #1466
- Fix archive graph not keeping node props by @ricopinazo in #1469
- Added import_node, nodes, edge, edges to rust+python with tests by @Haaroon in #1470
- Feature/last opened by @iamsmkr in #1471
- Bump aiohttp from 3.9.0 to 3.9.2 in /docs by @dependabot in #1478
- expose imporT_ops to graphwithdeletions in python by @Haaroon in #1475
- Made djikstras weight optional by @Haaroon in #1479
- Allow props to be returned as JSON & Upgrade rust to 1.75 by @Haaroon in #1482
- Shortest Path Algorithm in graphql by @rachchan in #1484
- Added small fix to stop edges returning self loops twice by @miratepuffin in #1485
- Added type property to node by @Haaroon in #1480
- Fixed slow edge layer lookup by @miratepuffin in #1487
- Bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in #1488
- Bump scrapy from 2.8.0 to 2.11.1 by @dependabot in #1489
- Updated benchmark to work with latest versions of libraries by @Haaroon in #1490
- make isolated components cohesive by adding edges in between them by @iamsmkr in #1494
- Bug/disable sccache by @Haaroon in #1496
- Moving Export.py into Rust by @Haaroon in #1495
- Allow user to change node_type by @Haaroon in #1497
- Document props by @ricopinazo in #1493
- impl log levels as config by @iamsmkr in #1498
- Improvements for the TimeIndexOps ergonomics and cleanup by @ljeub-pometry in #1500
- PageRank Args, is_self_loop on edge, remove all option, added bincode version on graph, fix all warns, by @Haaroon in #1499
- Fix panic on vectorised_graph.rs by @ricopinazo in #1502
- fix bug with node_type being overwritten by @Haaroon in #1507
- Require higher version of maturin by @louisch in #1508
- rename graph props to graph meta by @iamsmkr in #1501
- add test to verify fail if data not consistent in type by @iamsmkr in #1505
- Bring back caching and clean up dependency handling by @ljeub-pometry in #1509
- Remove the old caching by @ljeub-pometry in #1512
- GQL Simple Initial Benchmark by @Haaroon in #1513
- update base rust version by @Haaroon in #1517
- Fixed node_type in graphql by @miratepuffin in #1521
- move the commit out of the loop by @ljeub-pometry in #1523
- Added node_type to repr by @miratepuffin in #1522
- Feature/even more benchmarks by @Haaroon in #1525
- Fix/node type search by @miratepuffin in #1524
- Fixed issue with to_pyvis throwing an unwrap by @Haaroon in #1529
- Undirected shortest path by @rachchan in #1527
- add docs by @rachchan in #1532
- fix issue with sort by @Haaroon in #1533
- load_from_pandas is scrambling node order by @ljeub-pometry in #1535
- GraphQL save_graph now saves all props, edges and nodes correctly by @Haaroon in #1531
- GraphOps refactor by @ljeub-pometry in #1544
- Add the lost pip package matplotlib in the Python package by @haoxins in #1546
- internal property apis are not supposed to be public by @ljeub-pometry in #1547
- Moved/Improved g.nodes.to_df and g.edges.to_df and added support for python312 by @Haaroon in #1537
- fix issue with InnerTemopralGraph implementation and impl load from p… by @iamsmkr in #1543
- Improve strongly-connected components algorithm output by @ljeub-pometry in #1559
- Bump pillow from 10.2.0 to 10.3.0 in /docs by @dependabot in #1555
- Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #1556
- add more tests and singleton components output by @ljeub-pometry in #1560
- Bump transformers from 4.37.2 to 4.38.0 by @dependabot in #1563
- Feature/excludeapis by @iamsmkr in #1561
- Bump idna from 3.4 to 3.7 by @dependabot in #1566
- Feature/rename graphs by @iamsmkr in #1565
- convert nanoseconds py datetime to milliseconds by @iamsmkr in #1553
- Add largest connected component method by @D4rkisek in #1562
- Release v0.8.0 by @github-actions in #1568
- replace exclude layers with exclude valid layers to rid unwraps by @iamsmkr in #1569
- Minor 0.8.0 tweaks by @miratepuffin in #1572
New Contributors
**...
v0.7.0
Release v0.7.0
What's Changed
- Internal dispatcher by @miratepuffin in #1328
- Added failure case to internal dispatch by @miratepuffin in #1329
- Update internal_dispatch.yml by @Haaroon in #1330
- adding methods for layered edges in graphql by @rachchan in #1331
- Updated readme/examples by @miratepuffin in #1334
- Fixed bug where we do not reset the task runner by @miratepuffin in #1335
- Bump urllib3 from 1.26.17 to 1.26.18 by @dependabot in #1337
- Bump urllib3 from 1.26.17 to 1.26.18 in /docs by @dependabot in #1338
- Update rust_format_check.yml by @Haaroon in #1336
- Algorithm Result object complete rewrite, Add betweeen centrality, Add Zaks Karate Club by @Haaroon in #1327
- Modify AlgoResult.get to accept Strings in rust by @Haaroon in #1341
- move usecase to main package by @Haaroon in #1342
- Feature/in out shake it all about by @miratepuffin in #1345
- Extra Benchmarks for Algorithms using Large Generated Graphs by @Haaroon in #1344
- Adding methods for server side pagination by @rachchan in #1343
- added missing udpate constant proeprties from graphwithdeletions in p… by @Haaroon in #1350
- Added 'hard deletion' semantics by @miratepuffin in #1348
- More caching during CI building by @Haaroon in #1352
- Feature/more caching by @Haaroon in #1353
- Strongly Connected Components Algorithm by @iamsmkr in #1349
- Update test_python_workflow.yml by @Haaroon in #1359
- fix graphs with dels by @iamsmkr in #1358
- Multi document entities by @ricopinazo in #1339
- disabled code cov workflow by @Haaroon in #1364
- already existing properties must be updated by @iamsmkr in #1363
- fix edge_latest_time() in GraphWithDeletions and add a test by @ljeub-pometry in #1368
- Fix id strings in graphql by @rachchan in #1365
- Dont fail benchmark if major change by @Haaroon in #1372
- dont-fail-benchmark-on-alert by @Haaroon in #1373
- Bump aiohttp from 3.8.5 to 3.8.6 in /docs by @dependabot in #1376
- Feature/improve time ops by @ljeub-pometry in #1371
- Removed old examples, Created new monkey example by @Haaroon in #1347
- fix #327 by @iamsmkr in #1377
- Fix rust toolchain version to 1.73.0 by @Haaroon in #1378
- bump rtd rust version by @Haaroon in #1379
- Feature/python index graph by @miratepuffin in #1117
- Improvements for vectors python module by @ricopinazo in #1360
- save archive by @iamsmkr in #1381
- Adding some methods for the smart search dropdown button by @rachchan in #1386
- Add expansion filters by @ricopinazo in #1387
- LPA Algorithm rust+python+docs+tests by @Haaroon in #1390
- Efficient version for global motif counts by @narnolddd in #1322
- Bump aiohttp from 3.8.6 to 3.9.0 in /docs by @dependabot in #1395
- Bump twisted from 22.10.0 to 23.10.0 by @dependabot in #1397
- Feature/one hop filters by @ljeub-pometry in #1393
- fix the python test workflow by @ljeub-pometry in #1398
- Adding some missing graphql functions by @miratepuffin in #1392
- impl at and fix in/out degree by @iamsmkr in #1399
- fix property semantics for the graph with deletions by @ljeub-pometry in #1400
- fix wrong number of vertices returned by @iamsmkr in #1409
- Add graphql server handler and python wrapper for it by @ricopinazo in #1402
- pull out the EdgeStore functionality into trait EdgeLike by @fabianmurariu in #1405
- Vector docs and cache improvements by @ricopinazo in #1391
- Fix GraphQL queries with numbers and make embedding optional by @ricopinazo in #1410
- Syncing functionalty between graphql and python by @miratepuffin in #1401
- Vetrex 2 node by @miratepuffin in #1412
- Allow externally defined LockedLayeredIndex by @fabianmurariu in #1411
- Fix compilation for different feature combinations by @ljeub-pometry in #1416
- Fixed graphql docs by @miratepuffin in #1413
- event based temporal SEIR algorithm by @ljeub-pometry in #1415
- add search feature to graphql by @ljeub-pometry in #1417
- Fix save_graph GQL mutation so it does not create multiple files for the same graph name by @ljeub-pometry in #1419
- Feature/history datetime by @miratepuffin in #1420
- Added date_time history/items to Temporal properties by @miratepuffin in #1423
- Louvain algorithm by @ljeub-pometry in #1425
- Release v0.7.0 by @github-actions in #1427
Full Changelog: v0.6.1...v0.7.0