I'd be happy to be *shown* that I'm wrong.

ncmncm · on March 16, 2021

OK, hold my beer.

  #include <algorithm>
  #include <cctype>
  #include <iostream>
  #include <iterator>
  #include <string>
  #include <unordered_map>
  #include <vector>
  
  int main() {
    std::unordered_map<std::string, int>  counts;
    std::string word; word.reserve(1024);
    for (std::istreambuf_iterator<char>
        in(std::cin), end; in != end; ++in) {
      unsigned char c = *in;
      if (std::isspace(c)) {
        if (!word.empty()) {
          ++counts[word];
          word.clear();
        }
      } else word.push_back(std::tolower(c));
    }
    if (!word.empty()) {  // in case no EOL
      ++counts[word];
    }
    std::vector<std::pair<const std::string,int> const*>
      vec;
    vec.reserve(1<<15);
    for (auto const& word : counts) {
      vec.push_back(&word);
    }
    std::sort(vec.begin(), vec.end(),
      [](auto const* a, auto const* b) {
        return a->second > b->second;
      });
    for (auto const* word : vec) {
      std::cout << word->first
        << ' ' << word->second << '\n';
    }
  }

burntsushi · on March 16, 2021

Did you even bother to compare it to other programs? It's dog slow.

    $ hyperfine './optimized-ncmncm-cpp < kjvbible_x10.txt'
    Benchmark #1: ./optimized-ncmncm-cpp < kjvbible_x10.txt
      Time (mean ± σ):      1.396 s ±  0.023 s    [User: 1.381 s, System: 0.012 s]
      Range (min … max):    1.357 s …  1.429 s    10 runs

    $ hyperfine './optimized-cpp < kjvbible_x10.txt'
    Benchmark #1: ./optimized-cpp < kjvbible_x10.txt
      Time (mean ± σ):     407.6 ms ±   9.1 ms    [User: 400.3 ms, System: 6.3 ms]
      Range (min … max):   396.6 ms … 423.6 ms    10 runs

    $ hyperfine './optimized-c < kjvbible_x10.txt'
    Benchmark #1: ./optimized-c < kjvbible_x10.txt
      Time (mean ± σ):     203.6 ms ±   4.7 ms    [User: 195.1 ms, System: 8.0 ms]
      Range (min … max):   196.2 ms … 211.1 ms    14 runs

    $ hyperfine './rust/optimized/target/release/countwords < kjvbible_x10.txt'
    Benchmark #1: ./rust/optimized/target/release/countwords < kjvbible_x10.txt
      Time (mean ± σ):     314.6 ms ±   6.9 ms    [User: 302.0 ms, System: 12.1 ms]
      Range (min … max):   307.6 ms … 328.9 ms    10 runs

    $ hyperfine './rust/optimized-customhashmap/target/release/countwords < kjvbible_x10.txt'
    Benchmark #1: ./rust/optimized-customhashmap/target/release/countwords < kjvbible_x10.txt
      Time (mean ± σ):     239.0 ms ±   2.8 ms    [User: 231.0 ms, System: 7.5 ms]
      Range (min … max):   236.0 ms … 245.0 ms    12 runs

I compiled with

    g++ -O3 optimized-ncmncm.cpp -o optimized-ncmncm-cpp

I also tried with clang++ and got similarish results.

Care to retract any of your statements? Did you even bother to look at the programs before spouting off a bunch of nonsense here?

ncmncm · on March 16, 2021

I have no idea how you managed to get it to run that slow. I suspect you are running a different program.

burntsushi · on March 16, 2021

You suspect wrong.

ncmncm · on March 16, 2021

OK, you tell me why you get an order of magnitude slower than a VM on my aging laptop.

burntsushi · on March 16, 2021

I don't know. But Ben Hoyt got a similar time as me too. I saw your GitHub PR. Moreover, looking at your program, I would expect it to be slower than the other optimized versions (Rust, C, Go). Because it doesn't actually implement the optimizations in those programs! So not only is your code slow on two different compilers, but it also matches my prior.

You're the one who came out of the woodwork and said a bunch of nonsense. Maybe put in a little effort here to try to figure things out. Like, oh, I don't know, run the benchmarks on the other programs to compare relative timings. Like I did. Or show the commands you used to compile. Like I did. Or show the commands you used to run the program. Like I did. You want me to hold your beer? Fine. But put in the work.

My bet is that you're being just as sloppy now as you were in your comments. You're probably using the wrong input. See https://github.com/benhoyt/countwords/blob/aa45dcfd7eff177c8... and https://github.com/benhoyt/countwords/blob/aa45dcfd7eff177c8...