This study explores laminar flow, heat transfer, and mass transfer of a nanofluid in a porous medium using Buongiorno's two-phase model. The porous medium is modeled as a cavity with a non-uniform octagonal shape. The objective is to examine the effects of various parameters on hydrodynamic, thermal, and mass profiles, including Rayleigh number, Darcy number, Lewis number, Brownian motion, and thermophoresis parameters. A geometric contribution is introduced to enhance heat transfer rates within the porous cavity by corrugating the cold wall with varying wave numbers and amplitudes. The findings indicate that increasing the number of waves and their amplitude enhances the heat transfer rate, as reflected by a higher Nusselt number. At lower Rayleigh numbers, fluid movement emerges, and high permeability facilitates greater heat transfer and fluid flow velocities. However, dead zones develop at lower wave patterns, reducing heat transfer efficiency.