net/mlx5: Support lockless FTE read lookups

net/mlx5: Support lockless FTE read lookups

During connection tracking offloads with high number of connections,
(40K connections per second), flow table group lock contention is
observed.
To improve the performance by reducing lock contention, lockless
FTE read lookup is performed as described below.

Each flow table entry is refcounted.
Flow table entry is removed when refcount drops to zero.
rhash table allows rcu protected lookup.
Each hash table entry insertion and removal is write lock protected.

Hence, it is possible to perform lockless lookup in rhash table using
following scheme.

(a) Guard FTE entry lookup per group using rcu read lock.
(b) Before freeing the FTE entry, wait for all readers to finish
accessing the FTE.

Below example of one reader and write in parallel racing, shows
protection in effect with rcu lock.

lookup_fte_locked()
  rcu_read_lock();
  search_hash_table()
                                  existing_flow_group_write_lock();
                                  tree_put_node(fte)
                                    drop_ref_cnt(fte)
                                    del_sw_fte(fte)
                                    del_hash_table_entry();
                                    call_rcu();
                                  existing_flow_group_write_unlock();
  get_ref_cnt(fte) fails
  rcu_read_unlock();
                                  rcu grace period();
                                    [..]
                                    kmem_cache_free(fte);

Signed-off-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>