Van der Waals (vdW) materials consisting of two-dimensional (2D) building blocks have strong in-plane covalent bonding and weak interlayer interactions. While monolayer 2D materials exhibit impressive fracture resistance, as demonstrated in hexagonal boron nitride (h-BN), preserving these remarkable properties in vdW materials remains a challenge. Here we reveal an anomalous mechanical interlayer coupling that involves interlayer-friction toughening and edge-reconstruction embrittlement during the fracture of multilayer h-BN. Both asynchronous and synchronous fracture modes and their flaw-size dependence are identified. Edge reconstruction in the synchronous fracture mode can eliminate a toughening mechanism induced by lattice asymmetry in monolayer h-BN, leading to embrittlement of the multilayer h-BN, while the asynchronous fracture mode results in greater fracture resistance. Such findings will provide fundamental guidelines for engineering interlayer interactions in vdW materials including heterostructures and layered architectures for better mechanical and functional performances.