2.1.0 (#123)

# Patchnotes 2.1.0
The changes in this update are more than just "patches". With a new UI, a new feature, and a bunch of bug fixes, improvements and a new member on the dev team, we thought this was more of a minor update.

We would like to introduce @tsubasahane of MareCN to the team! We’re happy to work with them to bring Lightless and its features to the CN client as well as having another talented dev bring features and ideas to us. Speaking of which:

# Location Sharing (Big shout out to @tsubasahane for bringing this feature)

- Are you TIRED of scrambling to find the address of the venue you're in to share with your friends? We are introducing Location Sharing! An optional feature where you can share your location with direct pairs temporarily [30 minutes, 1 hour, 3 hours] minutes or until you turn it off for them. That's up to you! [#125](<#125>)  [#49](<Lightless-Sync/LightlessServer#49>)
- To share your location with a pair, click the three dots beside the pair and choose a duration to share with them. [#125](<#125>)  [#49](<Lightless-Sync/LightlessServer#49>)
- To view the location of someone who's shared with you, simply hover over the globe icon! [#125](<#125>)  [#49](<Lightless-Sync/LightlessServer#49>)

[1]

# Model Optimization (Mesh Decimating)
 - This new option can automatically “simplify” incoming character meshes to help performance by reducing triangle counts. You choose how strong the reduction is (default/recommended is 80%). [#131](<#131>)
 - Decimation only kicks in when a mesh is above a certain triangle threshold, and only for the items that qualify for it and you selected for. [#131](<#131>)
 - Hair meshes is always excluded, since simplifying hair meshes is very prone to breaking.
 - You can find everything under Settings → Performance → Model Optimization. [#131](<#131>)
+ ** IF YOU HAVE USED DECIMATION IN TESTING, PLEASE CLEAR YOUR CACHE ❗ **

[2]

# Animation (PAP) Validation (Safer animations)
 - Lightless now checks your currently animations to see if they work with your local skeleton/bone mod. If an animation matches, it’s included in what gets sent to other players. If it doesn’t, Lightless will skip it and write a warning to your log showing how many were skipped due to skeleton changes. Its defaulted to Unsafe (off). turn it on if you experience crashes from others users. [#131](<#131>)
 - Lightless also does the same kind of check for incoming animation files, to make sure they match the body/skeleton they were sent with. [#131](<#131>)
 - Because these checks can sometimes be a little picky, you can adjust how strict they are in Settings -> General -> Animation & Bones to reduce false positives. [#131](<#131>)

# UI Changes (Thanks to @kyuwu for UI Changes)
- The top part of the main screen has gotten a makeover. You can adjust the colors of the gradiant in the Color settings of Lightless. [#127](<#127>)

[3]

- Settings have gotten some changes as well to make this change more universal, and will use the same color settings. [#127](<#127>)
- The particle effects of the gradient are toggleable in 'Settings -> UI -> Behavior' [#127](<#127>)
- Instead of showing download/upload on bottom of Main UI, it will show VRAM usage and triangles with their optimization options next to it [#138](<#138>)

# LightFinder / ShellFinder
- UI Changes that follow our new design follow the color codes for the Gradient top as the main screen does.  [#127](<#127>)

[4]

Co-authored-by: defnotken <itsdefnotken@gmail.com>
Co-authored-by: azyges <aaaaaa@aaa.aaa>
Co-authored-by: cake <admin@cakeandbanana.nl>
Co-authored-by: Tsubasa <tsubasa@noreply.git.lightless-sync.org>
Co-authored-by: choco <choco@patat.nl>
Co-authored-by: celine <aaa@aaa.aaa>
Co-authored-by: celine <celine@noreply.git.lightless-sync.org>
Co-authored-by: Tsubasahane <wozaiha@gmail.com>
Co-authored-by: cake <cake@noreply.git.lightless-sync.org>
Reviewed-on: #123

LightlessSync/FileCache/CacheMonitor.cs

@@ -103,6 +103,7 @@ public sealed class CacheMonitor : DisposableMediatorSubscriberBase
     }
 
     record WatcherChange(WatcherChangeTypes ChangeType, string? OldPath = null);
+    private readonly record struct CacheEvictionCandidate(string FullPath, long Size, DateTime LastAccessTime);
     private readonly Dictionary<string, WatcherChange> _watcherChanges = new(StringComparer.OrdinalIgnoreCase);
     private readonly Dictionary<string, WatcherChange> _lightlessChanges = new(StringComparer.OrdinalIgnoreCase);
 
@@ -441,116 +442,40 @@ public sealed class CacheMonitor : DisposableMediatorSubscriberBase
             Logger.LogWarning(ex, "Could not determine drive size for storage folder {folder}", _configService.Current.CacheFolder);
         }
 
-        var files = Directory.EnumerateFiles(_configService.Current.CacheFolder)
-            .Select(f => new FileInfo(f))
-            .OrderBy(f => f.LastAccessTime)
-            .ToList();
-
+        var cacheFolder = _configService.Current.CacheFolder;
+        var candidates = new List<CacheEvictionCandidate>();
         long totalSize = 0;
-
-        foreach (var f in files)
-        {
-            token.ThrowIfCancellationRequested();
-
-            try
-            {
-                long size = 0;
-
-                if (!isWine)
-                {
-                    try
-                    {
-                        size = _fileCompactor.GetFileSizeOnDisk(f);
-                    }
-                    catch (Exception ex)
-                    {
-                        Logger.LogTrace(ex, "GetFileSizeOnDisk failed for {file}, using fallback length", f.FullName);
-                        size = f.Length;
-                    }
-                }
-                else
-                {
-                    size = f.Length;
-                }
-
-                totalSize += size;
-            }
-            catch (Exception ex)
-            {
-                Logger.LogTrace(ex, "Error getting size for {file}", f.FullName);
-            }
-        }
+        totalSize += AddFolderCandidates(cacheFolder, candidates, token, isWine);
+        totalSize += AddFolderCandidates(Path.Combine(cacheFolder, "downscaled"), candidates, token, isWine);
+        totalSize += AddFolderCandidates(Path.Combine(cacheFolder, "decimated"), candidates, token, isWine);
 
         FileCacheSize = totalSize;
 
-        if (Directory.Exists(_configService.Current.CacheFolder + "/downscaled"))
-        {
-            var filesDownscaled = Directory.EnumerateFiles(_configService.Current.CacheFolder + "/downscaled").Select(f => new FileInfo(f)).OrderBy(f => f.LastAccessTime).ToList();
-
-            long totalSizeDownscaled = 0;
-
-            foreach (var f in filesDownscaled)
-            {
-                token.ThrowIfCancellationRequested();
-
-                try
-                {
-                    long size = 0;
-
-                    if (!isWine)
-                    {
-                        try
-                        {
-                            size = _fileCompactor.GetFileSizeOnDisk(f);
-                        }
-                        catch (Exception ex)
-                        {
-                            Logger.LogTrace(ex, "GetFileSizeOnDisk failed for {file}, using fallback length", f.FullName);
-                            size = f.Length;
-                        }
-                    }
-                    else
-                    {
-                        size = f.Length;
-                    }
-
-                    totalSizeDownscaled += size;
-                }
-                catch (Exception ex)
-                {
-                    Logger.LogTrace(ex, "Error getting size for {file}", f.FullName);
-                }
-            }
-
-            FileCacheSize = (totalSize + totalSizeDownscaled);
-        }
-        else
-        {
-            FileCacheSize = totalSize;
-        }
-
         var maxCacheInBytes = (long)(_configService.Current.MaxLocalCacheInGiB * 1024d * 1024d * 1024d);
         if (FileCacheSize < maxCacheInBytes)
             return;
 
         var maxCacheBuffer = maxCacheInBytes * 0.05d;
 
-        while (FileCacheSize > maxCacheInBytes - (long)maxCacheBuffer && files.Count > 0)
+        candidates.Sort(static (a, b) => a.LastAccessTime.CompareTo(b.LastAccessTime));
+
+        var evictionTarget = maxCacheInBytes - (long)maxCacheBuffer;
+        var index = 0;
+        while (FileCacheSize > evictionTarget && index < candidates.Count)
         {
-            var oldestFile = files[0];
+            var oldestFile = candidates[index];
 
             try
             {
-                long fileSize = oldestFile.Length;
-                File.Delete(oldestFile.FullName);
-                FileCacheSize -= fileSize;
+                EvictCacheCandidate(oldestFile, cacheFolder);
+                FileCacheSize -= oldestFile.Size;
             }
             catch (Exception ex)
             {
-                Logger.LogTrace(ex, "Failed to delete old file {file}", oldestFile.FullName);
+                Logger.LogTrace(ex, "Failed to delete old file {file}", oldestFile.FullPath);
             }
 
-            files.RemoveAt(0);
+            index++;
         }
     }
 
@@ -559,6 +484,114 @@ public sealed class CacheMonitor : DisposableMediatorSubscriberBase
         HaltScanLocks.Clear();
     }
 
+    private long AddFolderCandidates(string directory, List<CacheEvictionCandidate> candidates, CancellationToken token, bool isWine)
+    {
+        if (!Directory.Exists(directory))
+        {
+            return 0;
+        }
+
+        long totalSize = 0;
+        foreach (var path in Directory.EnumerateFiles(directory))
+        {
+            token.ThrowIfCancellationRequested();
+
+            try
+            {
+                var file = new FileInfo(path);
+                var size = GetFileSizeOnDisk(file, isWine);
+                totalSize += size;
+                candidates.Add(new CacheEvictionCandidate(file.FullName, size, file.LastAccessTime));
+            }
+            catch (Exception ex)
+            {
+                Logger.LogTrace(ex, "Error getting size for {file}", path);
+            }
+        }
+
+        return totalSize;
+    }
+
+    private long GetFileSizeOnDisk(FileInfo file, bool isWine)
+    {
+        if (isWine)
+        {
+            return file.Length;
+        }
+
+        try
+        {
+            return _fileCompactor.GetFileSizeOnDisk(file);
+        }
+        catch (Exception ex)
+        {
+            Logger.LogTrace(ex, "GetFileSizeOnDisk failed for {file}, using fallback length", file.FullName);
+            return file.Length;
+        }
+    }
+
+    private void EvictCacheCandidate(CacheEvictionCandidate candidate, string cacheFolder)
+    {
+        if (TryGetCacheHashAndPrefixedPath(candidate.FullPath, cacheFolder, out var hash, out var prefixedPath))
+        {
+            _fileDbManager.RemoveHashedFile(hash, prefixedPath);
+        }
+
+        try
+        {
+            if (File.Exists(candidate.FullPath))
+            {
+                File.Delete(candidate.FullPath);
+            }
+        }
+        catch (Exception ex)
+        {
+            Logger.LogTrace(ex, "Failed to delete old file {file}", candidate.FullPath);
+        }
+    }
+
+    private static bool TryGetCacheHashAndPrefixedPath(string filePath, string cacheFolder, out string hash, out string prefixedPath)
+    {
+        hash = string.Empty;
+        prefixedPath = string.Empty;
+
+        if (string.IsNullOrEmpty(cacheFolder))
+        {
+            return false;
+        }
+
+        var fileName = Path.GetFileNameWithoutExtension(filePath);
+        if (string.IsNullOrEmpty(fileName) || !IsSha1Hash(fileName))
+        {
+            return false;
+        }
+
+        var relative = Path.GetRelativePath(cacheFolder, filePath)
+            .Replace(Path.AltDirectorySeparatorChar, Path.DirectorySeparatorChar);
+        var sanitizedRelative = relative.TrimStart(Path.DirectorySeparatorChar);
+        prefixedPath = Path.Combine(FileCacheManager.CachePrefix, sanitizedRelative);
+        hash = fileName;
+        return true;
+    }
+
+    private static bool IsSha1Hash(string value)
+    {
+        if (value.Length != 40)
+        {
+            return false;
+        }
+
+        foreach (var ch in value)
+        {
+            if (!Uri.IsHexDigit(ch))
+            {
+                return false;
+            }
+        }
+
+        return true;
+    }
+
     public void ResumeScan(string source)
     {
         if (!HaltScanLocks.ContainsKey(source)) HaltScanLocks[source] = 0;

LightlessSync/FileCache/ExternalCompactionExecutor.cs

@@ -0,0 +1,241 @@
+using Microsoft.Extensions.Logging;
+using System.Diagnostics;
+using System.IO.Pipes;
+using System.Text.Json;
+
+namespace LightlessSync.FileCache;
+
+internal sealed class ExternalCompactionExecutor : ICompactionExecutor, IDisposable
+{
+    private readonly ILogger<ExternalCompactionExecutor> _logger;
+    private readonly ICompactorContext _context;
+    private readonly TimeSpan _timeout = TimeSpan.FromMinutes(5);
+    private readonly string _pipeName;
+    private Process? _workerProcess;
+    private bool _disposed;
+    private readonly object _sync = new();
+
+    public ExternalCompactionExecutor(ILogger<ExternalCompactionExecutor> logger, ICompactorContext context)
+    {
+        _logger = logger;
+        _context = context;
+        _pipeName = $"LightlessCompactor-{Environment.ProcessId}";
+    }
+
+    public bool TryCompact(string filePath)
+    {
+        if (string.IsNullOrWhiteSpace(filePath) || !File.Exists(filePath))
+            return false;
+
+        if (!EnsureWorkerRunning())
+            return false;
+
+        try
+        {
+            var request = new CompactorRequest
+            {
+                Type = "compact",
+                Path = filePath
+            };
+
+            return SendRequest(request, out var response) && response?.Success == true;
+        }
+        catch (Exception ex)
+        {
+            _logger.LogWarning(ex, "External compactor failed for {file}", filePath);
+            return false;
+        }
+    }
+
+    public void Dispose()
+    {
+        if (_disposed)
+            return;
+
+        _disposed = true;
+
+        try
+        {
+            SendRequest(new CompactorRequest { Type = "shutdown" }, out _);
+        }
+        catch
+        {
+            // ignore
+        }
+
+        lock (_sync)
+        {
+            if (_workerProcess is null)
+                return;
+
+            TryKill(_workerProcess);
+            _workerProcess.Dispose();
+            _workerProcess = null;
+        }
+    }
+
+    private bool EnsureWorkerRunning()
+    {
+        lock (_sync)
+        {
+            if (_workerProcess is { HasExited: false })
+                return true;
+
+            _workerProcess?.Dispose();
+            _workerProcess = null;
+
+            var workerPath = ResolveWorkerPath();
+            if (string.IsNullOrEmpty(workerPath))
+                return false;
+
+            var args = BuildArguments();
+            var startInfo = new ProcessStartInfo
+            {
+                FileName = workerPath,
+                Arguments = args,
+                CreateNoWindow = true,
+                UseShellExecute = false,
+                RedirectStandardOutput = true,
+                RedirectStandardError = true
+            };
+
+            var process = new Process { StartInfo = startInfo };
+            if (!process.Start())
+                return false;
+
+            TrySetLowPriority(process);
+            _ = DrainAsync(process.StandardOutput, "stdout");
+            _ = DrainAsync(process.StandardError, "stderr");
+
+            _workerProcess = process;
+            return true;
+        }
+    }
+
+    private bool SendRequest(CompactorRequest request, out CompactorResponse? response)
+    {
+        response = null;
+        using var pipe = new NamedPipeClientStream(".", _pipeName, PipeDirection.InOut, PipeOptions.Asynchronous);
+
+        try
+        {
+            pipe.Connect((int)_timeout.TotalMilliseconds);
+        }
+        catch (Exception ex)
+        {
+            _logger.LogDebug(ex, "Compactor pipe connection failed.");
+            return false;
+        }
+
+        using var writer = new StreamWriter(pipe) { AutoFlush = true };
+        using var reader = new StreamReader(pipe);
+
+        var payload = JsonSerializer.Serialize(request);
+        writer.WriteLine(payload);
+
+        var readTask = reader.ReadLineAsync();
+        if (!readTask.Wait(_timeout))
+        {
+            _logger.LogWarning("Compactor pipe timed out waiting for response.");
+            return false;
+        }
+
+        var line = readTask.Result;
+        if (string.IsNullOrWhiteSpace(line))
+            return false;
+
+        try
+        {
+            response = JsonSerializer.Deserialize<CompactorResponse>(line);
+            return response is not null;
+        }
+        catch (Exception ex)
+        {
+            _logger.LogDebug(ex, "Failed to parse compactor response.");
+            return false;
+        }
+    }
+
+    private string? ResolveWorkerPath()
+    {
+        var baseDir = AppContext.BaseDirectory;
+        var exeName = OperatingSystem.IsWindows() || _context.IsWine
+            ? "LightlessCompactorWorker.exe"
+            : "LightlessCompactorWorker";
+        var path = Path.Combine(baseDir, exeName);
+        return File.Exists(path) ? path : null;
+    }
+
+    private string BuildArguments()
+    {
+        var args = new List<string> { "--pipe", Quote(_pipeName), "--parent", Environment.ProcessId.ToString() };
+        if (_context.IsWine)
+            args.Add("--wine");
+        return string.Join(' ', args);
+    }
+
+    private static string Quote(string value)
+    {
+        if (string.IsNullOrEmpty(value))
+            return "\"\"";
+
+        if (!value.Contains('"', StringComparison.Ordinal))
+            return "\"" + value + "\"";
+
+        return "\"" + value.Replace("\"", "\\\"", StringComparison.Ordinal) + "\"";
+    }
+
+    private static void TrySetLowPriority(Process process)
+    {
+        try
+        {
+            if (OperatingSystem.IsWindows())
+                process.PriorityClass = ProcessPriorityClass.BelowNormal;
+        }
+        catch
+        {
+            // ignore
+        }
+    }
+
+    private async Task DrainAsync(StreamReader reader, string label)
+    {
+        try
+        {
+            string? line;
+            while ((line = await reader.ReadLineAsync().ConfigureAwait(false)) != null)
+            {
+                if (_logger.IsEnabled(LogLevel.Trace))
+                    _logger.LogTrace("Compactor {label}: {line}", label, line);
+            }
+        }
+        catch
+        {
+            // ignore
+        }
+    }
+
+    private static void TryKill(Process process)
+    {
+        try
+        {
+            process.Kill(entireProcessTree: true);
+        }
+        catch
+        {
+            // ignore
+        }
+    }
+
+    private sealed class CompactorRequest
+    {
+        public string Type { get; init; } = "compact";
+        public string? Path { get; init; }
+    }
+
+    private sealed class CompactorResponse
+    {
+        public bool Success { get; init; }
+        public string? Error { get; init; }
+    }
+}

LightlessSync/FileCache/FileCacheManager.cs

@@ -27,6 +27,7 @@ public sealed class FileCacheManager : IHostedService
     private readonly ConcurrentDictionary<string, ConcurrentDictionary<string, FileCacheEntity>> _fileCaches = new(StringComparer.Ordinal);
     private readonly ConcurrentDictionary<string, FileCacheEntity> _fileCachesByPrefixedPath = new(StringComparer.OrdinalIgnoreCase);
     private readonly SemaphoreSlim _getCachesByPathsSemaphore = new(1, 1);
+    private readonly SemaphoreSlim _evictSemaphore = new(1, 1);
     private readonly Lock _fileWriteLock = new();
     private readonly IpcManager _ipcManager;
     private readonly ILogger<FileCacheManager> _logger;
@@ -114,6 +115,35 @@ public sealed class FileCacheManager : IHostedService
         return true;
     }
 
+    private static bool TryGetHashFromFileName(FileInfo fileInfo, out string hash)
+    {
+        hash = Path.GetFileNameWithoutExtension(fileInfo.Name);
+        if (string.IsNullOrWhiteSpace(hash))
+        {
+            return false;
+        }
+
+        if (hash.Length is not (40 or 64))
+        {
+            return false;
+        }
+
+        for (var i = 0; i < hash.Length; i++)
+        {
+            var c = hash[i];
+            var isHex = (c >= '0' && c <= '9')
+                || (c >= 'a' && c <= 'f')
+                || (c >= 'A' && c <= 'F');
+            if (!isHex)
+            {
+                return false;
+            }
+        }
+
+        hash = hash.ToUpperInvariant();
+        return true;
+    }
+
     private static string BuildVersionHeader() => $"{FileCacheVersionHeaderPrefix}{FileCacheVersion}";
 
     private static bool TryParseVersionHeader(string? line, out int version)
@@ -226,13 +256,23 @@ public sealed class FileCacheManager : IHostedService
             var compressed = LZ4Wrapper.WrapHC(raw, 0, raw.Length);
 
             var tmpPath = compressedPath + ".tmp";
-            await File.WriteAllBytesAsync(tmpPath, compressed, token).ConfigureAwait(false);
-            File.Move(tmpPath, compressedPath, overwrite: true);
+            try
+            {
+                await File.WriteAllBytesAsync(tmpPath, compressed, token).ConfigureAwait(false);
+                File.Move(tmpPath, compressedPath, overwrite: true);
+            }
+            finally
+            {
+                try { if (File.Exists(tmpPath)) File.Delete(tmpPath); } catch { /* ignore */ }
+            }
 
-            var compressedSize = compressed.LongLength;
+            var compressedSize = new FileInfo(compressedPath).Length;
             SetSizeInfo(hash, originalSize, compressedSize);
             UpdateEntitiesSizes(hash, originalSize, compressedSize);
 
+            var maxBytes = GiBToBytes(_configService.Current.MaxLocalCacheInGiB);
+            await EnforceCacheLimitAsync(maxBytes, token).ConfigureAwait(false);
+
             return compressed;
         }
         finally
@@ -277,9 +317,34 @@ public sealed class FileCacheManager : IHostedService
         _logger.LogTrace("Creating cache entry for {path}", path);
         var cacheFolder = _configService.Current.CacheFolder;
         if (string.IsNullOrEmpty(cacheFolder)) return null;
+        if (TryGetHashFromFileName(fi, out var hash))
+        {
+            return CreateCacheEntryWithKnownHash(fi.FullName, hash);
+        }
+
         return CreateFileEntity(cacheFolder, CachePrefix, fi);
     }
 
+    public FileCacheEntity? CreateCacheEntryWithKnownHash(string path, string hash)
+    {
+        if (string.IsNullOrWhiteSpace(hash))
+        {
+            return CreateCacheEntry(path);
+        }
+
+        FileInfo fi = new(path);
+        if (!fi.Exists) return null;
+        _logger.LogTrace("Creating cache entry for {path} using provided hash", path);
+        var cacheFolder = _configService.Current.CacheFolder;
+        if (string.IsNullOrEmpty(cacheFolder)) return null;
+        if (!TryBuildPrefixedPath(fi.FullName, cacheFolder, CachePrefix, out var prefixedPath, out _))
+        {
+            return null;
+        }
+
+        return CreateFileCacheEntity(fi, prefixedPath, hash);
+    }
+
     public FileCacheEntity? CreateFileEntry(string path)
     {
         FileInfo fi = new(path);
@@ -562,9 +627,10 @@ public sealed class FileCacheManager : IHostedService
         }
     }
 
-    public void RemoveHashedFile(string hash, string prefixedFilePath)
+    public void RemoveHashedFile(string hash, string prefixedFilePath, bool removeDerivedFiles = true)
     {
         var normalizedPath = NormalizePrefixedPathKey(prefixedFilePath);
+        var removedHash = false;
 
         if (_fileCaches.TryGetValue(hash, out var caches))
         {
@@ -577,11 +643,16 @@ public sealed class FileCacheManager : IHostedService
 
             if (caches.IsEmpty)
             {
-                _fileCaches.TryRemove(hash, out _);
+                removedHash = _fileCaches.TryRemove(hash, out _);
             }
         }
 
         _fileCachesByPrefixedPath.TryRemove(normalizedPath, out _);
+
+        if (removeDerivedFiles && removedHash)
+        {
+            RemoveDerivedCacheFiles(hash);
+        }
     }
 
     public void UpdateHashedFile(FileCacheEntity fileCache, bool computeProperties = true)
@@ -597,7 +668,8 @@ public sealed class FileCacheManager : IHostedService
             fileCache.Hash = Crypto.ComputeFileHash(fileCache.ResolvedFilepath, Crypto.HashAlgo.Sha1);
             fileCache.LastModifiedDateTicks = fi.LastWriteTimeUtc.Ticks.ToString(CultureInfo.InvariantCulture);
         }
-        RemoveHashedFile(oldHash, prefixedPath);
+        var removeDerivedFiles = !string.Equals(oldHash, fileCache.Hash, StringComparison.OrdinalIgnoreCase);
+        RemoveHashedFile(oldHash, prefixedPath, removeDerivedFiles);
         AddHashedFile(fileCache);
     }
 
@@ -747,7 +819,7 @@ public sealed class FileCacheManager : IHostedService
     {
         try
         {
-            RemoveHashedFile(fileCache.Hash, fileCache.PrefixedFilePath);
+            RemoveHashedFile(fileCache.Hash, fileCache.PrefixedFilePath, removeDerivedFiles: false);
             var extensionPath = fileCache.ResolvedFilepath.ToUpper(CultureInfo.InvariantCulture) + "." + ext;
             File.Move(fileCache.ResolvedFilepath, extensionPath, overwrite: true);
             var newHashedEntity = new FileCacheEntity(fileCache.Hash, fileCache.PrefixedFilePath + "." + ext, DateTime.UtcNow.Ticks.ToString(CultureInfo.InvariantCulture));
@@ -764,6 +836,33 @@ public sealed class FileCacheManager : IHostedService
         }
     }
 
+    private void RemoveDerivedCacheFiles(string hash)
+    {
+        var cacheFolder = _configService.Current.CacheFolder;
+        if (string.IsNullOrWhiteSpace(cacheFolder))
+        {
+            return;
+        }
+
+        TryDeleteDerivedCacheFile(Path.Combine(cacheFolder, "downscaled", $"{hash}.tex"));
+        TryDeleteDerivedCacheFile(Path.Combine(cacheFolder, "decimated", $"{hash}.mdl"));
+    }
+
+    private void TryDeleteDerivedCacheFile(string path)
+    {
+        try
+        {
+            if (File.Exists(path))
+            {
+                File.Delete(path);
+            }
+        }
+        catch (Exception ex)
+        {
+            _logger.LogTrace(ex, "Failed to delete derived cache file {path}", path);
+        }
+    }
+
     private void AddHashedFile(FileCacheEntity fileCache)
     {
         var normalizedPath = NormalizePrefixedPathKey(fileCache.PrefixedFilePath);
@@ -877,6 +976,83 @@ public sealed class FileCacheManager : IHostedService
         }, token).ConfigureAwait(false);
     }
 
+    private async Task EnforceCacheLimitAsync(long maxBytes, CancellationToken token)
+    {
+        if (string.IsNullOrWhiteSpace(CacheFolder) || maxBytes <= 0) return;
+
+        await _evictSemaphore.WaitAsync(token).ConfigureAwait(false);
+        try
+        {
+            Directory.CreateDirectory(CacheFolder);
+
+            foreach (var tmp in Directory.EnumerateFiles(CacheFolder, "*" + _compressedCacheExtension + ".tmp"))
+            {
+                try { File.Delete(tmp); } catch { /* ignore */ }
+            }
+
+            var files = Directory.EnumerateFiles(CacheFolder, "*" + _compressedCacheExtension, SearchOption.TopDirectoryOnly)
+                .Select(p => new FileInfo(p))
+                .Where(fi => fi.Exists)
+                .OrderBy(fi => fi.LastWriteTimeUtc)
+                .ToList();
+
+            long total = files.Sum(f => f.Length);
+            if (total <= maxBytes) return;
+
+            foreach (var fi in files)
+            {
+                token.ThrowIfCancellationRequested();
+                if (total <= maxBytes) break;
+
+                var hash = Path.GetFileNameWithoutExtension(fi.Name);
+
+                try
+                {
+                    var len = fi.Length;
+                    fi.Delete();
+                    total -= len;
+                    _sizeCache.TryRemove(hash, out _);
+                }
+                catch (Exception ex)
+                {
+                    _logger.LogWarning(ex, "Failed to evict cache file {file}", fi.FullName);
+                }
+            }
+        }
+        finally
+        {
+            _evictSemaphore.Release();
+        }
+    }
+
+    private static long GiBToBytes(double gib)
+    {
+        if (double.IsNaN(gib) || double.IsInfinity(gib) || gib <= 0)
+            return 0;
+
+        var bytes = gib * 1024d * 1024d * 1024d;
+
+        if (bytes >= long.MaxValue) return long.MaxValue;
+
+        return (long)Math.Round(bytes, MidpointRounding.AwayFromZero);
+    }
+
+    private void CleanupOrphanCompressedCache()
+    {
+        if (string.IsNullOrWhiteSpace(CacheFolder) || !Directory.Exists(CacheFolder))
+            return;
+
+        foreach (var path in Directory.EnumerateFiles(CacheFolder, "*" + _compressedCacheExtension))
+        {
+            var hash = Path.GetFileNameWithoutExtension(path);
+            if (!_fileCaches.ContainsKey(hash))
+            {
+                try { File.Delete(path); }
+                catch (Exception ex) { _logger.LogWarning(ex, "Failed deleting orphan {file}", path); }
+            }
+        }
+    }
+
     public async Task StartAsync(CancellationToken cancellationToken)
     {
         _logger.LogInformation("Starting FileCacheManager");
@@ -1060,6 +1236,8 @@ public sealed class FileCacheManager : IHostedService
             {
                 await WriteOutFullCsvAsync(cancellationToken).ConfigureAwait(false);
             }
+
+            CleanupOrphanCompressedCache();
         }
 
         _logger.LogInformation("Started FileCacheManager");

LightlessSync/FileCache/PluginCompactorContext.cs

@@ -0,0 +1,20 @@
+using LightlessSync.LightlessConfiguration;
+using LightlessSync.Services;
+
+namespace LightlessSync.FileCache;
+
+internal sealed class PluginCompactorContext : ICompactorContext
+{
+    private readonly LightlessConfigService _configService;
+    private readonly DalamudUtilService _dalamudUtilService;
+
+    public PluginCompactorContext(LightlessConfigService configService, DalamudUtilService dalamudUtilService)
+    {
+        _configService = configService;
+        _dalamudUtilService = dalamudUtilService;
+    }
+
+    public bool UseCompactor => _configService.Current.UseCompactor;
+    public string CacheFolder => _configService.Current.CacheFolder;
+    public bool IsWine => _dalamudUtilService.IsWine;
+}

LightlessSync/FileCache/TransientResourceManager.cs

@@ -25,7 +25,6 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
     private readonly object _ownedHandlerLock = new();
     private readonly string[] _handledFileTypes = ["tmb", "pap", "avfx", "atex", "sklb", "eid", "phyb", "scd", "skp", "shpk", "kdb"];
     private readonly string[] _handledRecordingFileTypes = ["tex", "mdl", "mtrl"];
-    private readonly string[] _handledFileTypesWithRecording;
     private readonly HashSet<GameObjectHandler> _playerRelatedPointers = [];
     private readonly object _playerRelatedLock = new();
     private readonly ConcurrentDictionary<nint, GameObjectHandler> _playerRelatedByAddress = new();
@@ -42,8 +41,6 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
         _dalamudUtil = dalamudUtil;
         _actorObjectService = actorObjectService;
         _gameObjectHandlerFactory = gameObjectHandlerFactory;
-        _handledFileTypesWithRecording = _handledRecordingFileTypes.Concat(_handledFileTypes).ToArray();
-
         Mediator.Subscribe<PenumbraResourceLoadMessage>(this, Manager_PenumbraResourceLoadEvent);
         Mediator.Subscribe<ActorTrackedMessage>(this, msg => HandleActorTracked(msg.Descriptor));
         Mediator.Subscribe<ActorUntrackedMessage>(this, msg => HandleActorUntracked(msg.Descriptor));
@@ -297,7 +294,7 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
 
     private void DalamudUtil_FrameworkUpdate()
     {
-        RefreshPlayerRelatedAddressMap();
+        _ = Task.Run(() => RefreshPlayerRelatedAddressMap());
 
         lock (_cacheAdditionLock)
         {
@@ -306,20 +303,64 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
 
         if (_lastClassJobId != _dalamudUtil.ClassJobId)
         {
-            _lastClassJobId = _dalamudUtil.ClassJobId;
-            if (SemiTransientResources.TryGetValue(ObjectKind.Pet, out HashSet<string>? value))
-            {
-                value?.Clear();
-            }
-
-            PlayerConfig.JobSpecificCache.TryGetValue(_dalamudUtil.ClassJobId, out var jobSpecificData);
-            SemiTransientResources[ObjectKind.Player] = PlayerConfig.GlobalPersistentCache.Concat(jobSpecificData ?? []).ToHashSet(StringComparer.OrdinalIgnoreCase);
-            PlayerConfig.JobSpecificPetCache.TryGetValue(_dalamudUtil.ClassJobId, out var petSpecificData);
-            SemiTransientResources[ObjectKind.Pet] = new HashSet<string>(
-                petSpecificData ?? [],
-                StringComparer.OrdinalIgnoreCase);
+            UpdateClassJobCache();
         }
 
+        CleanupAbsentObjects();
+    }
+
+    private void RefreshPlayerRelatedAddressMap()
+    {
+        var tempMap = new ConcurrentDictionary<nint, GameObjectHandler>();
+        var updatedFrameAddresses = new ConcurrentDictionary<nint, ObjectKind>();
+
+        lock (_playerRelatedLock)
+        {
+            foreach (var handler in _playerRelatedPointers)
+            {
+                var address = (nint)handler.Address;
+                if (address != nint.Zero)
+                {
+                    tempMap[address] = handler;
+                    updatedFrameAddresses[address] = handler.ObjectKind;
+                }
+            }
+        }
+
+        _playerRelatedByAddress.Clear();
+        foreach (var kvp in tempMap)
+        {
+            _playerRelatedByAddress[kvp.Key] = kvp.Value;
+        }
+
+        _cachedFrameAddresses.Clear();
+        foreach (var kvp in updatedFrameAddresses)
+        {
+            _cachedFrameAddresses[kvp.Key] = kvp.Value;
+        }
+    }
+
+    private void UpdateClassJobCache()
+    {
+        _lastClassJobId = _dalamudUtil.ClassJobId;
+        if (SemiTransientResources.TryGetValue(ObjectKind.Pet, out HashSet<string>? value))
+        {
+            value?.Clear();
+        }
+
+        PlayerConfig.JobSpecificCache.TryGetValue(_dalamudUtil.ClassJobId, out var jobSpecificData);
+        SemiTransientResources[ObjectKind.Player] = PlayerConfig.GlobalPersistentCache
+            .Concat(jobSpecificData ?? [])
+            .ToHashSet(StringComparer.OrdinalIgnoreCase);
+
+        PlayerConfig.JobSpecificPetCache.TryGetValue(_dalamudUtil.ClassJobId, out var petSpecificData);
+        SemiTransientResources[ObjectKind.Pet] = new HashSet<string>(
+            petSpecificData ?? [],
+            StringComparer.OrdinalIgnoreCase);
+    }
+
+    private void CleanupAbsentObjects()
+    {
         foreach (var kind in Enum.GetValues(typeof(ObjectKind)).Cast<ObjectKind>())
         {
             if (!_cachedFrameAddresses.Any(k => k.Value == kind) && TransientResources.Remove(kind, out _))
@@ -349,26 +390,6 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
         _semiTransientResources = null;
     }
 
-    private void RefreshPlayerRelatedAddressMap()
-    {
-        _playerRelatedByAddress.Clear();
-        var updatedFrameAddresses = new ConcurrentDictionary<nint, ObjectKind>();
-        lock (_playerRelatedLock)
-        {
-            foreach (var handler in _playerRelatedPointers)
-            {
-                var address = (nint)handler.Address;
-                if (address != nint.Zero)
-                {
-                    _playerRelatedByAddress[address] = handler;
-                    updatedFrameAddresses[address] = handler.ObjectKind;
-                }
-            }
-        }
-
-        _cachedFrameAddresses = updatedFrameAddresses;
-    }
-
     private void HandleActorTracked(ActorObjectService.ActorDescriptor descriptor)
     {
         if (descriptor.IsInGpose)
@@ -499,46 +520,51 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
 
     private void Manager_PenumbraResourceLoadEvent(PenumbraResourceLoadMessage msg)
     {
+        var gamePath = msg.GamePath.ToLowerInvariant();
         var gameObjectAddress = msg.GameObject;
-        if (!_cachedFrameAddresses.TryGetValue(gameObjectAddress, out var objectKind))
-        {
-            if (_actorObjectService.TryGetOwnedKind(gameObjectAddress, out var ownedKind))
-            {
-                objectKind = ownedKind;
-            }
-            else
-            {
-                return;
-            }
-        }
-
-        var gamePath = NormalizeGamePath(msg.GamePath);
-        if (string.IsNullOrEmpty(gamePath))
-        {
-            return;
-        }
+        var filePath = msg.FilePath;
 
         // ignore files already processed this frame
+        if (_cachedHandledPaths.Contains(gamePath)) return;
+
         lock (_cacheAdditionLock)
         {
-            if (!_cachedHandledPaths.Add(gamePath))
-            {
-                return;
-            }
+            _cachedHandledPaths.Add(gamePath);
+        }
+
+        // replace individual mtrl stuff
+        if (filePath.StartsWith("|", StringComparison.OrdinalIgnoreCase))
+        {
+            filePath = filePath.Split("|")[2];
+        }
+        // replace filepath
+        filePath = filePath.ToLowerInvariant().Replace("\\", "/", StringComparison.OrdinalIgnoreCase);
+
+        // ignore files that are the same
+        var replacedGamePath = gamePath.ToLowerInvariant().Replace("\\", "/", StringComparison.OrdinalIgnoreCase);
+        if (string.Equals(filePath, replacedGamePath, StringComparison.OrdinalIgnoreCase))
+        {
+            return;
         }
 
         // ignore files to not handle
-        var handledTypes = IsTransientRecording ? _handledFileTypesWithRecording : _handledFileTypes;
-        if (!HasHandledFileType(gamePath, handledTypes))
+        var handledTypes = IsTransientRecording ? _handledRecordingFileTypes.Concat(_handledFileTypes) : _handledFileTypes;
+        if (!handledTypes.Any(type => gamePath.EndsWith(type, StringComparison.OrdinalIgnoreCase)))
         {
+            lock (_cacheAdditionLock)
+            {
+                _cachedHandledPaths.Add(gamePath);
+            }
             return;
         }
 
-        var filePath = NormalizeFilePath(msg.FilePath);
-
-        // ignore files that are the same
-        if (string.Equals(filePath, gamePath, StringComparison.Ordinal))
+        // ignore files not belonging to anything player related
+        if (!_cachedFrameAddresses.TryGetValue(gameObjectAddress, out var objectKind))
         {
+            lock (_cacheAdditionLock)
+            {
+                _cachedHandledPaths.Add(gamePath);
+            }
             return;
         }
 
@@ -553,12 +579,13 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
         _playerRelatedByAddress.TryGetValue(gameObjectAddress, out var owner);
         bool alreadyTransient = false;
 
-        bool transientContains = transientResources.Contains(gamePath);
-        bool semiTransientContains = SemiTransientResources.Values.Any(value => value.Contains(gamePath));
+        bool transientContains = transientResources.Contains(replacedGamePath);
+        bool semiTransientContains = SemiTransientResources.SelectMany(k => k.Value)
+            .Any(f => string.Equals(f, gamePath, StringComparison.OrdinalIgnoreCase));
         if (transientContains || semiTransientContains)
         {
             if (!IsTransientRecording)
-                Logger.LogTrace("Not adding {replacedPath} => {filePath}, Reason: Transient: {contains}, SemiTransient: {contains2}", gamePath, filePath,
+                Logger.LogTrace("Not adding {replacedPath} => {filePath}, Reason: Transient: {contains}, SemiTransient: {contains2}", replacedGamePath, filePath,
                     transientContains, semiTransientContains);
             alreadyTransient = true;
         }
@@ -566,10 +593,10 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
         {
             if (!IsTransientRecording)
             {
-                bool isAdded = transientResources.Add(gamePath);
+                bool isAdded = transientResources.Add(replacedGamePath);
                 if (isAdded)
                 {
-                    Logger.LogDebug("Adding {replacedGamePath} for {gameObject} ({filePath})", gamePath, owner?.ToString() ?? gameObjectAddress.ToString("X"), filePath);
+                    Logger.LogDebug("Adding {replacedGamePath} for {gameObject} ({filePath})", replacedGamePath, owner?.ToString() ?? gameObjectAddress.ToString("X"), filePath);
                     SendTransients(gameObjectAddress, objectKind);
                 }
             }
@@ -577,7 +604,7 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
 
         if (owner != null && IsTransientRecording)
         {
-            _recordedTransients.Add(new TransientRecord(owner, gamePath, filePath, alreadyTransient) { AddTransient = !alreadyTransient });
+            _recordedTransients.Add(new TransientRecord(owner, replacedGamePath, filePath, alreadyTransient) { AddTransient = !alreadyTransient });
         }
     }
 
@@ -676,4 +703,4 @@ public sealed class TransientResourceManager : DisposableMediatorSubscriberBase
     {
         public bool AddTransient { get; set; }
     }
-}
+}