You need to identify what is doing that. Do you have slugs/snails? Have you seen different insects flitting about? What, if anything, have you done to learn about which insects are the good ones and which are the bad ones?

To know what to do you need to know what is causing the problem.

kimmq is kimmsr

Welcome to the GardenWeb, Igor. What is your general location? It helps us to know that.

How carefully have you searched for culprits? Taking a flashlight out at night is a good way to catch slugs, snails, and even caterpillars. Rose slug larvae hide underneath the leaves

What is the yellow plant? It reminds me of a member of the Brassicaceae family, being devoured by caterpillars.

Looked with flashlight but did not find the culprit. Is there a good insecticide (preferably non-toxic) to get rid of both slugs/caterpillars?

Before applying any insecticide you need to know what you are targeting. Integrated Pest Management is a 5 step method of problem solving.

1. Identify what the problem is.

2. Identify what is causing the problem.

3 Apply the least toxic method of control.

4. Review to see if that solved the problem.

5. Return to step 1.

You are still at step 2.

If you simply want the name of some poison to spray around, one that most likely will also kill off the beneficial insects, there are other forums for that.

kimmq is kimmsr.

Could be. Are you in the
United States and could you take that insect to the local office of your state universities Cooperative Extension Service for identification?

kimmq is kimmsr.

That looks like a Vine Weevil (if OP is in Europe). They chew notches out of the edge of leaves and their larvae are a menace in potted plant soil, eating the roots. However, the damage in the second picture above does not look like VW because the holes are not around the edges of the leaves. If not VW another weevil, maybe rose specific since I notice both pictures of edge damage are on roses.

That is a weevil showing on the leaf in the last photo and vine weevils/root weevils are quite common in the US as well and are virtually omnipresent here in the PNW. The larvae seldom do any significant damage to the roots of most plants (young, containerized plants may be the exception) but the adults will disfigure the leaves of a lot of plant types by chewing notches out of the leaf edges. The good news is that this is only cosmetic damage - it doesn't harm the plants. Typically most folks in my area just learn to live with the leaf damage as controlling the weevils to any significant degree involves routine applications of some serious chemical pesticides. And is usually only a temporary fix at best.

You do have other insect damage showing as well but need to find and identify the culprit first before you can successfully address the problem. Or if it is even worth addressing - a lot of foliar damage resulting from insects is just cosmetic in nature and doesn't actually affect the growth of the plant......just its appearance.