There are 2 types of quests. NPC quests which you can complete when you interact with NPCs and your game quests which can be found inside your inventory, the "!" button at the top.
Completing quests will get you EXP as well as rewards such as diamonds. The first few quests are just collecting resources quests to get you warmed up. The first true quest begins when you are required to kill the WORM.
The WORM is a dangerous monster if you are not prepared, and it can be found in the first level of the caves.
Remember, complete your quests!