Why are DataFrames considered more efficient than RDDs?

DataFrames are considered more efficient than RDDs primarily because they include optimizations for query execution. This means that DataFrames format data in a way that allows Apache Spark to better optimize the execution of queries using techniques such as Catalyst optimization and Tungsten execution engine. Catalyst, for instance, is able to apply various optimization rules automatically, allowing for better execution plans, while Tungsten improves memory management and code generation.

These optimizations enable DataFrames to leverage underlying features like whole-stage code generation, which can significantly enhance performance by reducing the amount of Java Virtual Machine (JVM) overhead and optimizing data access patterns. This level of optimization is not available with RDDs, which lack the same level of abstraction and cannot take advantage of optimization techniques for query execution.

The other options do not accurately capture the main efficiency benefits of DataFrames over RDDs. For example, while finer control over data partitions and manual data manipulation can be features associated with RDDs, they do not inherently contribute to the efficiency in query execution that DataFrames provide. Furthermore, DataFrames can be created from various sources, not just SQL queries, which makes option C incorrect.