Re: PSA: New intel MDS vulnerability mitigations cause measurableslowdown

Hi,

On 2019-05-14 15:30:52 -0700, Andres Freund wrote:
> There's a new set of CPU vulnerabilities, so far only affecting intel
> CPUs. Cribbing from the linux-kernel announcement I'm referring to
> https://xenbits.xen.org/xsa/advisory-297.html
> for details.
> 
> The "fix" is for the OS to perform some extra mitigations:
> https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html
> https://www.kernel.org/doc/html/latest/x86/mds.html#mds
> 
> *And* SMT/hyperthreading needs to be disabled, to be fully safe.
> 
> Fun.
> 
> I've run a quick pgbench benchmark:
> 
> *Without* disabling SMT, for readonly pgbench, I'm seeing regressions
> between 7-11%, depending on the size of shared_buffers (and some runtime
> variations).  That's just on my laptop, with an i7-6820HQ / Haswell CPU.
> I'd be surprised if there weren't adversarial loads with bigger
> slowdowns - what gets more expensive with the mitigations is syscalls.

The profile after the mitigations looks like:

+    3.62%  postgres         [kernel.vmlinux]         [k] do_syscall_64
+    2.99%  postgres         postgres                 [.] _bt_compare
+    2.76%  postgres         postgres                 [.] hash_search_with_hash_value
+    2.33%  postgres         [kernel.vmlinux]         [k] entry_SYSCALL_64
+    1.69%  pgbench          [kernel.vmlinux]         [k] do_syscall_64
+    1.61%  postgres         postgres                 [.] AllocSetAlloc
     1.41%  postgres         postgres                 [.] PostgresMain
+    1.22%  pgbench          [kernel.vmlinux]         [k] entry_SYSCALL_64
+    1.11%  postgres         postgres                 [.] LWLockAcquire
+    0.86%  postgres         postgres                 [.] PinBuffer
+    0.80%  postgres         postgres                 [.] LockAcquireExtended
+    0.78%  postgres         [kernel.vmlinux]         [k] psi_task_change
     0.76%  pgbench          pgbench                  [.] threadRun
     0.69%  postgres         postgres                 [.] LWLockRelease
+    0.69%  postgres         postgres                 [.] SearchCatCache1
     0.66%  postgres         postgres                 [.] LockReleaseAll
+    0.65%  postgres         postgres                 [.] GetSnapshotData
+    0.58%  postgres         postgres                 [.] hash_seq_search
     0.54%  postgres         postgres                 [.] hash_search
+    0.53%  postgres         [kernel.vmlinux]         [k] __switch_to
+    0.53%  postgres         postgres                 [.] hash_any
     0.52%  pgbench          libpq.so.5.12            [.] pqParseInput3
     0.50%  pgbench          [kernel.vmlinux]         [k] do_raw_spin_lock

where do_syscall_64 show this instruction profile:

       │     static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
       │     {
       │             asm_volatile_goto("1:"
  1.58 │     ↓ jmpq   bd
       │     mds_clear_cpu_buffers():
       │              * Works with any segment selector, but a valid writable
       │              * data segment is the fastest variant.
       │              *
       │              * "cc" clobber is required because VERW modifies ZF.
       │              */
       │             asm volatile("verw %[ds]" : : [ds] "m" (ds) : "cc");
 77.38 │       verw   0x13fea53(%rip)        # ffffffff82400ee0 <ds.4768>
       │     do_syscall_64():
       │             }
       │
       │             syscall_return_slowpath(regs);
       │     }
 13.18 │ bd:   pop    %rbx
  0.08 │       pop    %rbp
       │     ← retq
       │                     nr = syscall_trace_enter(regs);
       │ c0:   mov    %rbp,%rdi
       │     → callq  syscall_trace_enter


Where verw is the instruction that was recycled to now have the
side-effect of flushing CPU buffers.

Greetings,

Andres Freund