IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled. #10766
Conversation
# Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java
| @@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool | |||
| try { | |||
| lock.tryLock(1); | |||
|
|
|||
| try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) { | |||
| Set<File> delete = new HashSet<>(); | |||
| Long lostSgmnt = findLastLostSegement(); | |||
There was a problem hiding this comment.
lostSgmnt -> lastSegBeforeSkip
findLastLostSegement -> findLastSegmentBeforeSkip
Be aware about typo in segEment in method name.
|
|
||
| AtomicLong lastSgmnt = new AtomicLong(-1); | ||
| if (lostSgmnt != null) | ||
| deleteAll(lostSgmnt); |
There was a problem hiding this comment.
deleteAll -> deleteAllUntil or
deleteAllPrevious(lostSgmnt + 1);
|
|
||
| AtomicLong lastSgmnt = new AtomicLong(-1); | ||
| if (lostSgmnt != null) |
There was a problem hiding this comment.
if (seg != null) -> this check can be moved to the method.
It seem, that will simplify the code.
| .reversed()) // Sort by segment index. | ||
| .forEach(path -> { | ||
| long idx = FileWriteAheadLogManager.segmentIndex(path); | ||
| Long cdcDisableSgmnt = findLastCdcDisableSegment(); |
There was a problem hiding this comment.
findLastCdcDisableSegment -> findLastSegmentWithCdcDisabledRecord.
cdcDisableSgmnt -> lastSegWithDisabledRec
| } | ||
| if (lostSgmnt != null || cdcDisableSgmnt != null) | ||
| resetWalState(); | ||
| else { |
There was a problem hiding this comment.
brace can be omitted
else if (log.isDebugEnabled)
log.debug("Here!");
| } | ||
|
|
||
| /** */ | ||
| private void resetWalState() { |
| @@ -515,8 +521,18 @@ public void setFileIOFactory(FileIOFactory ioFactory) { | |||
| name, oldVal, newVal)); | |||
| } | |||
|
|
|||
| if (newVal != null && newVal) | |||
| if (newVal != null && newVal) { | |||
There was a problem hiding this comment.
A few linew below you check both values - oldVal && !newVal.
Here you check only newVal.
Does it mean that disable record can be recorded twice? When oldVal == newVal
So, it seems you should keep logic of both check consistent.
| if (!inMemoryCdc || rec instanceof CdcDisableRecord || !cdcDisabled.getOrDefault(false)) | ||
| return false; | ||
|
|
||
| LT.warn(log, "Logging CDC data records to WAL skipped. The '" + CDC_DISABLED + |
There was a problem hiding this comment.
LT.warn effectively creates an object to check timeout of logging on every invocation
IgniteBiTuple<Class<? extends Throwable>, String> tup =
e != null && !byMsg ? F.<Class<? extends Throwable>, String>t(e.getClass(), e.getMessage()) :
F.<Class<? extends Throwable>, String>t(null, longMsg);
So we shouldn't call it for every DataRecord that will be logged on every entry change.
Let's log changes of cdcDisabled state only.
|
SonarCloud Quality Gate failed.
|
…led.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
The Contribution Checklist
The description explains WHAT and WHY was made instead of HOW.
The following pattern must be used:
IGNITE-XXXX Change summarywhereXXXX- number of JIRA issue.(see the Maintainers list)
the
green visaattached to the JIRA ticket (see TC.Bot: Check PR)Notes
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com #ignite channel.