The subset of users can be selected (a) using k-most similar users or (b) a set of users whose similarity values are above a certain predefined threshold.

changed to

The subset of users can be selected choosing (a) the k most similar users or (b) users with similarity values above a certain predefined threshold.

There were multiple problems with this sentence; first “using,” even if it would have been a proper word, was inside the first condition, and thus the second condition was disconnected from the clause that introduced the conditions. It was also a bit not clear in the sentence whether the sets have similarity values or the users, till you looked at the subject/verb agreement: users … are above. Of course, logic would also dictate the same, but why should the reader stop or hesitate anywhere? “k-most” was almost as good as “the k most”, but then we don’t usually say “three-most ordinary articles”; we say “the three most ordinary articles” or “the three most-ordinary articles” (which I would prefer, but somehow is rarely used). I made the “k” italic since it’s a variable, but that’s OK; different people have different conventions, just follow one within one work. I would still debate whether “certain … threshold” doesn’t amount to redundance, but I would leave the author at peace here: maybe his book would be 30% of what it was originally if I start paring down the redundancies!