Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IGNITE-23222 Restore incremental snapshot fails if consistensId is not a String #11545

Open
wants to merge 112 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
0ccbae3
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 20, 2023
afd36c3
Merge pull request #1 from apache/master
Nastya828 Sep 20, 2023
1af61bd
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 21, 2023
0f7992f
Merge branch 'master' into ignite-20426
Sep 22, 2023
c9c6996
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 22, 2023
5c6a76c
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 22, 2023
3681d43
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 22, 2023
c6e7140
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 22, 2023
1ee8dc4
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 25, 2023
8e6d137
IGNITE-19880 Fix negative duration in the SQL query system view
Sep 26, 2023
7a99b2c
Merge pull request #2 from Nastya828/ignite-20426
Nastya828 Sep 26, 2023
5c02c35
Merge branch 'master' into ignite-19880
Sep 26, 2023
aa30338
IGNITE-19880 Fix negative duration in the SQL query system view
Sep 27, 2023
ca15cd1
IGNITE-19880 Fix negative duration in the SQL query system view
Sep 27, 2023
61d9d8d
Merge remote-tracking branch 'upstream/master'
Sep 27, 2023
32e0b55
Merge branch 'master' into ignite-19880
Sep 27, 2023
76539a8
Merge pull request #3 from Nastya828/ignite-19880
Nastya828 Sep 29, 2023
279a7ba
Merge branch 'apache:master' into master
Nastya828 Sep 29, 2023
0b63d6b
Merge remote-tracking branch 'upstream/master'
Oct 31, 2023
8563f58
IGNITE-20761
Oct 31, 2023
2a67313
IGNITE-20761 Reflection
Nov 1, 2023
38a163a
IGNITE-20761 desc in DistributedProprty
Nov 2, 2023
bfbdd79
IGNITE-20761 annotation deleted
Nov 2, 2023
cd16796
Merge remote-tracking branch 'upstream/master'
Nov 2, 2023
e78d0b4
ignite-20761 add info
Nov 3, 2023
efc8cbf
Merge remote-tracking branch 'upstream/master' into IGNITE-20761
Nov 7, 2023
5c2a565
IGNITE-20761 descriprions for distributed properties
Nov 7, 2023
58072de
Merge remote-tracking branch 'upstream/master'
Nov 7, 2023
00ba028
IGNITE-20761 Enum properties
Nov 7, 2023
65a685d
IGNITE-20761 checks descriptions
Nov 8, 2023
a8d1971
IGNITE-20761 fixed descriptions
Nov 8, 2023
1483000
IGNITE-20761 fixed snapshot dsc
Nov 8, 2023
b342aef
IGNITE-20761 fixed sql dsc
Nov 8, 2023
71b4798
IGNITE-20761 fixed sql dsc
Nov 8, 2023
7dabcf1
IGNITE-20761 test fix
Nov 9, 2023
02fa9c1
IGNITE-20761 add sorted in job result
Nov 9, 2023
500793b
Merge pull request #4 from Nastya828/IGNITE-20761
Nastya828 Nov 9, 2023
48edecd
Merge remote-tracking branch 'upstream/master'
Nov 9, 2023
f0ae7fd
Merge remote-tracking branch 'upstream/master'
Nov 15, 2023
6f29b4d
Merge remote-tracking branch 'upstream/master'
Nov 20, 2023
a880f0e
Merge remote-tracking branch 'upstream/master'
Nov 24, 2023
2d6c34a
Merge remote-tracking branch 'upstream/master'
Nov 24, 2023
b311564
Merge remote-tracking branch 'upstream/master'
Nov 27, 2023
d514b3f
IGNITE-20926 Added 2.16 release notes. (#11070)
NSAmelchev Nov 29, 2023
75f7b09
IGNITE-20748 Updated java opens (#11030)
IgGusev Nov 30, 2023
57eedc2
IGNITE-21008 Updated Netty dependency to 4.1.101.Final version (#11074)
nao-it Dec 4, 2023
15fb7b6
IGNITE-10268 Removed documentation about deprecated "replicatedOnly" …
liyuj Dec 5, 2023
6cb7b02
IGNITE-21015 Added the security subject ID to the cluster state chang…
NSAmelchev Dec 5, 2023
21e878c
IGNITE-20614 Add rate limiter for dumps (#11076)
timoninmaxim Dec 6, 2023
9e362be
IGNITE-21021 Removed ADMIN_OPS permissions check for cache operations…
petrov-mg Dec 6, 2023
e6934af
IGNITE-21031 SQL Calcite: Fix performance statistics failure on neste…
alex-plekhanov Dec 7, 2023
72348fc
IGNITE-19436 Fixed jvmdefaults.bat on JDK 17 (#10707)
liyuj Dec 7, 2023
4670f9c
IGNITE-21029 Updated log4j2 dependency to 2.22.0 version (#11080)
nao-it Dec 7, 2023
ddbccef
IGNITE-17700 Introduce CdcManager (#11044)
timoninmaxim Dec 7, 2023
cab5997
IGNITE-21013 Support encryption of cache dumps (#11078)
nizhikov Dec 8, 2023
0208de3
[MINOR] Fix error message in test
nizhikov Dec 8, 2023
24357c2
IGNITE-21042 Move CreateDumpFutureTask#toLong to IgniteUtils (#11048)
timoninmaxim Dec 8, 2023
17eaff3
IGNITE-21041 Make FilteredRecord local for RecordSerializer (#11082)
timoninmaxim Dec 8, 2023
eba2ca9
IGNITE-17700 Refactoring WalRecordsConsumer (#11050)
timoninmaxim Dec 8, 2023
154ed03
IGNITE-21046 Move binaryMetadataUpdateListener to basic interface (#1…
timoninmaxim Dec 8, 2023
c3895b0
IGNITE-14823 Abbrevation checkstyle rule (#9153)
nizhikov Dec 9, 2023
0de43d3
IGNITE-20426 Sync deprecated methods in the ClientCacheConfiguration
Sep 22, 2023
5af1fdb
IGNITE-20761
Oct 31, 2023
27b65f5
IGNITE-20761 Reflection
Nov 1, 2023
a9b96e8
IGNITE-20761 desc in DistributedProprty
Nov 2, 2023
bf0c3b4
IGNITE-20761 annotation deleted
Nov 2, 2023
a5ef5ed
IGNITE-20761 fixed sql dsc
Nov 8, 2023
dd9b8bb
IGNITE-20761 fixed sql dsc
Nov 8, 2023
c68c444
IGNITE-20761 fixed sql dsc
Nov 8, 2023
24d99fc
IGNITE-20761 fixed sql dsc
Nov 8, 2023
e64577f
IGNITE-20761 fixed sql dsc
Nov 8, 2023
7c0c062
IGNITE-20761 fixed sql dsc
Nov 8, 2023
46d51c1
IGNITE-20761 fixed sql dsc
Nov 8, 2023
d7f88d6
IGNITE-14823 Affinity abbrevation (#11083)
nizhikov Dec 11, 2023
870168e
IGNITE-14823 Argument abbrevation (#11085)
nizhikov Dec 11, 2023
624a0dd
IGNITE-21056 Use thread local buffer for encrypted dump (#11086)
yurinaryshkin Dec 12, 2023
3f4168e
IGNITE-14823 Argument abbrevation (#11088)
nizhikov Dec 12, 2023
79dfac7
IGNITE-14823 Buffer abbrevation (#11089)
nizhikov Dec 12, 2023
f64b653
IGNITE-14823 Class abbrevation (#11092)
nizhikov Dec 13, 2023
105788b
IGNITE-14823 Command abbrevation (#11093)
nizhikov Dec 13, 2023
b046e7c
Documentation: Add ARM64 to the ISA list in prerequisites
ptupitsyn Dec 13, 2023
bc9482c
IGNITE-21078 .NET: Fix platform cache update with negative cache ID -…
alex-plekhanov Dec 14, 2023
bd5467a
IGNITE-14823 Config abbrevation (#11095)
nizhikov Dec 14, 2023
2e93fbe
IGNITE-14823 Context abbrevation (#11096)
nizhikov Dec 14, 2023
8bfc4ec
IGNITE-21032 Ensured all entries in AttributeList are Attributes (#11…
simon-greatrix Dec 14, 2023
2159ecd
IGNITE-21085 Fixed the update versions script fails on ignite-calcite…
NSAmelchev Dec 15, 2023
be55186
IGNITE-21096 Fix standalone examples compilation (#11103)
NSAmelchev Dec 15, 2023
53e50ca
IGNITE-20761 fixed sql dsc
Nov 8, 2023
c0e5d06
IGNITE-20761 fixed sql dsc
Nov 8, 2023
634a57c
IGNITE-20761 fixed sql dsc
Nov 8, 2023
9f72da0
IGNITE-20761 fixed sql dsc
Nov 8, 2023
63aaf2e
IGNITE-20761 fixed sql dsc
Nov 8, 2023
c5c7c8a
IGNITE-20761 fixed sql dsc
Nov 8, 2023
33073f5
IGNITE-20761 fixed sql dsc
Nov 8, 2023
5466149
IGNITE-20761 fixed sql dsc
Nov 8, 2023
6ad63f1
Merge branch 'apache:master' into master
Nastya828 Dec 18, 2023
c18de2d
Merge branch 'apache:master' into master
Nastya828 Dec 18, 2023
e089bda
Merge branch 'apache:master' into master
Nastya828 Dec 19, 2023
f5098fa
Merge branch 'apache:master' into master
Nastya828 Dec 20, 2023
9c10ad8
Merge branch 'apache:master' into master
Nastya828 Dec 21, 2023
4d71a40
Merge branch 'apache:master' into master
Nastya828 Dec 22, 2023
33728cb
Merge branch 'apache:master' into master
Nastya828 Mar 24, 2024
3075d26
Merge branch 'apache:master' into master
Nastya828 Mar 26, 2024
0082699
Merge branch 'apache:master' into master
Nastya828 Aug 15, 2024
76b1cc0
Merge branch 'apache:master' into master
Nastya828 Sep 18, 2024
e5a60b4
Merge branch 'apache:master' into master
Nastya828 Sep 19, 2024
e2753b9
Merge branch 'apache:master' into master
Nastya828 Sep 20, 2024
6e4960e
IGNITE-23222 Restore incremental snapshot
Sep 19, 2024
65038ab
IGNITE-23222 remove test
Sep 20, 2024
c749d86
IGNITE-23222 fixed test
Sep 20, 2024
b126915
IGNITE-23222 fixed branch
Sep 20, 2024
960104e
IGNITE-23222 fixed checkstyle
Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
import org.apache.ignite.compute.ComputeJobResultPolicy;
import org.apache.ignite.compute.ComputeTaskAdapter;
import org.apache.ignite.internal.IgniteEx;
import org.apache.ignite.internal.processors.cache.GridCacheSharedContext;
import org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor;
import org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager;
import org.apache.ignite.internal.processors.cache.persistence.wal.reader.IgniteWalIteratorFactory;
Expand All @@ -49,7 +50,9 @@
import org.apache.ignite.resources.LoggerResource;
import org.jetbrains.annotations.NotNull;

import static java.lang.String.valueOf;
import static org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager.incrementalSnapshotWalsDir;
import static org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager.snapshotMetaFileName;

/** Snapshot task to verify snapshot metadata on the baseline nodes for given snapshot name. */
@GridInternal
Expand Down Expand Up @@ -112,7 +115,7 @@ public MetadataVerificationJob(SnapshotMetadataVerificationTaskArg arg) {

if (arg.incrementIndex() > 0) {
List<SnapshotMetadata> metas = snpMeta.stream()
.filter(m -> m.consistentId().equals(ignite.localNode().consistentId()))
.filter(m -> m.consistentId().equals(valueOf(ignite.localNode().consistentId())))
.collect(Collectors.toList());

if (metas.size() != 1) {
Expand Down Expand Up @@ -174,7 +177,9 @@ private void checkMeta(SnapshotMetadata meta) {
/** Checks that all incremental snapshots are present, contain correct metafile and WAL segments. */
public void checkIncrementalSnapshots(SnapshotMetadata fullMeta, SnapshotMetadataVerificationTaskArg arg) {
try {
IgniteSnapshotManager snpMgr = ignite.context().cache().context().snapshotMgr();
GridCacheSharedContext<Object, Object> ctx = ignite.context().cache().context();

IgniteSnapshotManager snpMgr = ctx.snapshotMgr();

// Incremental snapshot must contain ClusterSnapshotRecord.
long startSeg = fullMeta.snapshotRecordPointer().index();
Expand All @@ -187,7 +192,9 @@ public void checkIncrementalSnapshots(SnapshotMetadata fullMeta, SnapshotMetadat
"[snpName=" + arg.snapshotName() + ", snpPath=" + arg.snapshotPath() + ", incrementIndex=" + inc + ']');
}

String metaFileName = IgniteSnapshotManager.snapshotMetaFileName(ignite.localNode().consistentId().toString());
String folderName = ctx.kernalContext().pdsFolderResolver().resolveFolders().folderName();

String metaFileName = snapshotMetaFileName(folderName);

File metafile = incSnpDir.toPath().resolve(metaFileName).toFile();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteException;
import org.apache.ignite.cache.QueryEntity;
import org.apache.ignite.cluster.ClusterState;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
Expand Down Expand Up @@ -440,6 +441,42 @@ public void testStagesFail() throws Exception {
assertCacheKeys(srv.cache(DEFAULT_CACHE_NAME), CACHE_KEYS_RANGE);
}

/** */
@Test
public void testDefaultConsistentId() throws Exception {
assumeFalse("https://issues.apache.org/jira/browse/IGNITE-17819", encryption);

IgniteEx ignite = startGrid(getConfiguration().setConsistentId(null));

ignite.cluster().state(ClusterState.ACTIVE);

IgniteCache<Integer, Integer> cache = ignite.getOrCreateCache(new CacheConfiguration<>(DEFAULT_CACHE_NAME));

cache.put(1, 1);

ignite.snapshot().createSnapshot(SNAPSHOT_NAME).get();

cache.put(2, 2);

ignite.snapshot().createIncrementalSnapshot(SNAPSHOT_NAME).get();

ignite.destroyCache(DEFAULT_CACHE_NAME);

awaitPartitionMapExchange();

ignite.snapshot().restoreSnapshot(SNAPSHOT_NAME, F.asList(DEFAULT_CACHE_NAME)).get(getTestTimeout());

assertEquals(1, cache.size());

ignite.destroyCache(DEFAULT_CACHE_NAME);

awaitPartitionMapExchange();

ignite.snapshot().restoreSnapshot(SNAPSHOT_NAME, F.asList(DEFAULT_CACHE_NAME), 1).get(getTestTimeout());

assertEquals(2, cache.size());
}

/** */
private void checkFailWhenCacheDestroyed(String cache2rvm, String errMsg) throws Exception {
IgniteEx srv = startGridsWithCache(
Expand Down