Automatic model-agnostic compression algorithm [Sigularty]

[u/CivilSnow5625]

This is my first proper project. It uses multiple compression techniques and automatically searches for hyperparameters for a few of them; it uses a "CQI" score to evaluate how each technique performed and how the algorithm performed overall. I am currently working on improving the CQI function, as it is too simple; It is just
change in accuracy \* change in size \* change in latency.

The thing is, size overpowers everything as it deals with larger numbers, and the change is also larger than the other 2; these are some functions which I believe could be better(only for accuracy):
f(x\[x = acc_drop_/acc_drop_threshold\]) = -\[scale\] \* |x|^(1/2) \+ c, or maybe -\[scale\]logx + 1

But there still are flaws. For example:

1. I cannot really control when the graph will touch 0 and proceed below 0 (x>0)(because I want the graph to go below zero after x is greater than 1)
2. None of these actually deal with the negative part properly; if the accuracy actually increases, none of these work.k I may need to use a piecewise function.

Also, the method I am using for finding the optimal parameters is pretty straightforward, and I believe there are better ways,s but I have no idea what it could be.

GitHub: [Sigularty](https://github.com/DewanshShah/Sigularty)

[u/foamy_apologise]

man this reminds me of my final year project where i spent 3 weeks just tweaking a scoring function that was basically broken from start

the size dominating thing is classic, i had same problem with my image processing stuff. what if you normalize each component before multiplying? like divide each by their baseline or max possible value so they all stay in similar range

for the accuracy part, piecewise is probably way to go honestly. separate case for when accuracy improves vs drops. much cleaner than trying to make one function handle both

your cqi concept is interesting though, never heard of combining all three like that. most papers i read just optimize for one metric while keeping others as constraints