Preheat an oven. Place potatoes in a medium pot, then cover with cool water. Set over high heat, bring to a boil, then cook until tender. Drain, cool, then transfer to a parchment lined baking sheet, leaving a few inches.
Place a piece of parchment paper on top of the potatoes, then use a large cup to press down firmly, smashing each potato individually. Remove the top piece of parchment paper, discard, then drizzle potatoes with olive oil, Kosher salt, and black pepper.
Transfer to the preheated oven and roast until the edges are crispy, but the centers remain tender and fluffy.
In a small bowl, whisk together extra virgin olive oil, fresh lemon juice, lemon zest, chopped shallot, chopped capers, and chopped dill. Stir well.
Drizzle the caper sauce over the potatoes, then garnish with feta cheese and freshly cracked black pepper. Serve immediately!