Erdős problem 32

Does there exist a set $A \subseteq \mathbb{N}$ such that $|A \cap \{1, \ldots, N\}| = o((\log N)^2)$ and every sufficiently large integer can be written as $p + a$ for some prime $p$ and $a \in A$?