Force applied to A will be equal to tension in the string which is equal to Mg.

There can be 2 cases for the motion of the blocks

One is block A moving over block B & other is both the blocks move together.

1st case : let block A move over B

Applying COE on block A

$\sum F_x=Mg-0.4N_1=0 ------(i)\\ \sum F_y=N_1-23g=0 ----- (ii) $

Solving 2 eqns

We get $M=9.2 kg$

2nd case : let blocks A & B move together

Applying COE on block B

$\sum F_x=Mg-0.2 N=0 ---- (i)\\ \sum F_y=N-23g-26g =0 ------ (ii)$

Solving 2 eqns

We get $M=11.8 kg$

As in case 1 required force for impending motion is smaller, case 1 will take place at impending motion.

Hence, maximum mass M is 9.2 kg