ODTQA-FoRe: An Open-Domain Tabular Question Answering Dataset for Future Data Forecasting and Reasoning

Early table question answering (TQA) datasets, such as WikiTableQuestions Pasupat and Liang (2015), focused on complex queries but provided textual answers only. Spider Yu et al. (2018), while evaluating generalization on complex SQL queries, relies on structured outputs, limiting its open-domain applicability. These datasets are primarily domain-specific.

Recent advancements extend TQA to open-domain scenarios. NQ-TABLES Herzig et al. (2021) introduces candidate table retrieval, and Open-WikiTable Kweon et al. (2023) enhances this with metadata. RETQA Wang et al. (2025b) explores open-domain TQA in real estate, leveraging spoken language understanding (SLU) annotations Xing et al. (2025); Qin et al. (2025) for improved retrieval. FutureX Zeng et al. (2025b) addresses future-oriented QA, focusing on qualitative reasoning rather than structured data. In contrast, ODTQA-FoRe targets precise time-series forecasting over structured tables, a largely overlooked area in open-domain TQA.

Recent approaches leverage large language models (LLMs) for TQA tasks Fang et al. (2024). OPENTAB Kong et al. (2024) integrates open-domain knowledge for table-based fact-checking, while SLUTQA Wang et al. (2025b) enhances retrieval and SQL generation accuracy through SLU Cheng et al. (2023). However, these methods mainly focus on historical query answering rather than future predictions.

Independent applications of LLMs for forecasting include TP-BERTa Yan et al. (2024), which discretizes numerical features but does not handle sequential predictions, and LLMTIME Gruver et al. (2023b), which employs pretrained LLMs for time series without TQA integration.

Thus, existing methods predominantly address historical queries, non-sequential forecasting, or time series predictions, lacking a unifying framework for forecasting-oriented TQA. Our work fills this gap, representing the first effort in Open-Domain Tabular Question Answering for Future Data Forecasting and Reasoning.

The dataset for this study originates from the RETQA dataset Wang et al. (2025b), which covers land auctions, real estate project sales, and corporate finance. Our work focuses on the real estate sales data, initially spanning eight major Chinese cities for the year 2022. We significantly expand this by leveraging the original data source (http://www.fangdi.com.cn/) to include two additional cities, Tianjin and Chengdu, and extend the timeframe to cover January 2022 to December 2024. This results in a comprehensive dataset of 60,183 real estate projects, each with up to 36 months of data.

To prepare the data for our task, we first define a temporal split: December 31, 2023, serves as the reference date, with data from 2022–2023 designated as historical observations and data from 2024 treated as future ground truth, which is strictly withheld during model inference. Given the inherent sparsity in real estate records, we then apply stringent filtering criteria, retaining only projects with at least six months of sales data in the historical period (2022–2023) and at least one record in the forecast period (2024). After applying these criteria, we obtain a refined dataset of 11,149 projects for a subsequent QA pair generation.

To prevent data leakage and ensure a fair evaluation, we first partition the 11,149 filtered projects into training, validation, and test subsets. This project-level split is performed using a 6:2:2 ratio, resulting in 6,562 projects for the training set, 2,278 for the validation set, and 2,309 for the test set. This partitioning guarantees that there is no overlap of projects between the sets. The QA pairs for each split are subsequently generated exclusively from their corresponding project subsets.

The data corresponding to all 11,149 refined projects is then used to construct the database environments. Since the original RETQA data source is in Chinese, this process primarily involves data cleaning and localization, where all Chinese column headers, city names, district names, and project names are translated into English. The protocols for translation and schema standardization are detailed in Appendix A.1. After this unified preparation, the dataset is split by time. The historical data (2022–2023) is aggregated by city, district, and year into 288 tables. These tables are then imported into a PostgreSQL Stonebraker et al. (2019) database to serve as the agent’s queryable knowledge base, each renamed following the format project_price_table_{district}_{city}_{y- ear} (e.g., project_price_table_zengcheng_d- istrict_guangzhou_2022). Separately, the future data (2024) is organized into 144 tables using the same aggregation and naming convention and stored in an isolated database named “future prices” to facilitate the automated execution of ground truth SQL queries during evaluation (see Appendix A.2 for details). Note that throughout these databases, tables do not distinguish between projects from the training, validation, or test sets, as the agent cannot access the full database contents and can only interact with it through a text-to-SQL interface.

To support both time series forecasting and forecast-based reasoning tasks, we design 26 distinct sets of seed templates, comprising 7 templates specifically for forecasting tasks and 19 for reasoning tasks. Each set includes a natural language question template, a historical data SQL template, and a target (future) data SQL template. These templates primarily incorporate four variables: city, district, project name, and time. City, district, and project name variables are populated by randomly sampling from the database, while the time variable is determined based on actual data availability for the respective projects in 2024, ensuring alignment with months that have valid price records.

The forecasting templates generate queries requiring predictions of specific statistical indicators (e.g., price, averages, or extreme values) over predefined periods (see Figure 1, Q2). Conversely, reasoning templates address more complex analytical tasks, including comparisons of future price trends among multiple projects or identification of projects satisfying specific criteria (see Figure 1, Q3).

Prior to large-scale generation, we initially create ten QA pairs per template set, totaling 260 pairs. These pairs undergo rigorous manual review to ensure grammatical fluency, semantic clarity, correctness, and efficiency of SQL queries, as well as accuracy and conciseness of results. Any identified issues prompt iterative revisions until all QA pairs pass validation. Subsequently, large-scale automated QA pair generation proceeds. For more details on the types of templates, please refer to Appendix A.2.

The final ODTQA-FoRe dataset includes, for each QA instance: a natural language question, either a natural language answer (for reasoning queries) or a numerical answer (for forecasting queries), an SQL query to retrieve historical data, and an SQL query for retrieving future data labels. Historical data query results provide necessary context, while future data query results serve as evaluation labels. Answers are objective, uniquely determined by their corresponding questions. For numerical prediction, we provide only directly relevant values; for reasoning, only the direct conclusion. Multiple values or conclusions, if applicable, are presented as structured lists. Examples and detailed answer formats are provided in Appendix A.2. All four components (question, answer, historical data SQL, and future data SQL) are generated simultaneously, ensuring precise alignment.

As detailed in Section 3.2, each template utilizes four primary variables: city, district, project name, and time. During QA generation, cities and districts are randomly sampled first, followed by selecting project names from their corresponding dataset splits (training, validation, or test).

To handle temporal references, we create 98 natural language expressions denoting future periods (e.g., “next month,” “next year,” “next quarter,” “first half of the year”), each mapped to precise month lists (e.g., “next quarter” corresponds to [January 2024, February 2024, March 2024]). For each project, we identify all months in 2024 with available records, selecting appropriate expressions fully covered by the data. For instance, “next quarter” is used only if data for January to March 2024 is fully available. Subsequently, one eligible expression is randomly selected for template filling. Historical data SQL queries cover all months from 2022–2023, while future data SQL queries explicitly include months in 2024 with available records.

Using this template-driven approach, we initially generate 36,400 QA pairs. After removing duplicates, invalid queries, and empty results, we finalize 28,507 unique QA pairs, divided according to project assignments into 16,944 training pairs, 5,742 validation pairs, and 5,821 testing pairs. Additionally, following previous practices Wang et al. (2025b), we employ LLMs to rewrite original questions enhancing dataset diversity and realism. Comprehensive details on the rewriting procedure and dataset statistics are provided in Appendix A.3 and A.4.

In this section, we present TimeFore, a collaborative framework designed for Open-Domain Tabular Question Answering involving Future Data Forecasting and Reasoning (ODTQA-FoRe). As illustrated in Figure 2, TimeFore decomposes the task across three specialized agents. The Retriever handles data acquisition by first identifying relevant data tables and then generating a SQL query to extract historical information. Subsequently, the Forecaster leverages function-calling to invoke external, specialized models for accurate time-series prediction. Finally, the Analyzer synthesizes the retrieved data with the generated forecast to construct a precise and consistently formatted final answer.

In open-domain scenarios, the performance of accurately retrieving relevant data from a vast repository based on a user’s query is paramount, as this step directly governs the quality of all downstream tasks. To address this, the Retriever agent executes a two-stage process. The process commences with table retrieval, where the agent leverages an LLM’s summarization capability by employing few-shot prompting with five examples to convert the user query into a concise, canonical text in the style of a table caption. This text is first used to attempt a direct match with existing captions in the database. Should this attempt fail, the BM25 algorithm Robertson and Walker (1994) is then employed to retrieve the most semantically relevant table.

Upon securing the target table, the agent proceeds to generate the appropriate SQL query. This is again accomplished through in-context learning, guided by a system prompt containing five distinct exemplars. Each exemplar consists of a natural language question, its corresponding ground-truth table caption, and the target SQL query. To ensure robustness, these examples are sampled to cover diverse query types (e.g., single and multi-item). The LLM is prompted with the user’s original question and the identified table caption to generate an initial SQL query. To ensure robustness and address potential syntax or logic errors, particularly in complex scenarios, the Retriever agent employs an execution-feedback loop, utilizing the sqlQueryTool to execute the SQL within the database. If execution fails, the agent refines the SQL based on the error feedback. This loop continues until success is achieved or a maximum of 25 iterations is reached, ensuring reliable data retrieval. A detailed illustration of the prompt design is available in Appendix E.

Upon receiving the SQL results, the Forecaster interprets the serialized triples formatted as [project name, year–month, price] and converts them into a numerical list for the forecasting tools Liang et al. (2025). This historical input, limited to a 24-month window, remains compact enough to fit within the LLM’s context limits, thus avoiding token overflow issues and ensuring smooth data integration for forecasting.

While LLMs possess extensive general knowledge, their native forecasting capabilities for time-series tasks often lag behind specialized models without task-specific fine-tuning Gruver et al. (2023a); Tan et al. (2024). The Forecaster agent is designed to bridge this gap. Consistent with the TimeFore framework’s strategy, it leverages the LLM’s function-calling capability to orchestrate specialized, high-performance time-series models, thereby addressing the LLM’s inherent predictive weaknesses.

The process begins once the Forecaster receives the historical data sequences from the Retriever. Its core action is to invoke a predefined function, the imputationThenPredictionTool. This function encapsulates a sophisticated forecasting pipeline: it first validates and cleans the input time series, then employs the TimesNet model Wu et al. (2023) for robust data imputation. Based on this complete and imputed sequence, the TimeXer model Wang et al. (2024b) is then used to generate a price trend forecast for the subsequent 12 months. The tool concludes by returning the results as a structured three-tuple, containing the project name, year-month, and predicted price.

This architecture creates a powerful synergy: the LLM excels at orchestrating the workflow and understanding the task context, while the specialized models handle the precise, mathematical heavy-lifting of forecasting. This division of labor significantly enhances the overall forecasting accuracy and reliability.

For operational simplicity, the Forecaster is configured to generate a full 12-month forecast for the year 2024. This complete forecast is then passed to the Analyzer agent, which is responsible for reasoning upon these results to extract and deliver the specific answers pertinent to the user’s original query.

The Analyzer is the final-stage agent in the TimeFore framework, responsible for synthesizing the inputs from the preceding agents into a precise, standardized, and user-centric final answer. Its primary function is to correctly interpret the user’s ultimate goal and apply the appropriate reasoning strategy. To this end, the Analyzer first employs a BERT-based classifier to categorize the user’s query as either direct time-series forecasting (e.g., "What will the price be in June 2024?") or forecast-based reasoning (e.g., "Will the price in June exceed the price in January?"). Based on this classification, a distinct and tailored prompt, enriched with five illustrative examples via in-context learning, is selected to guide the LLM’s response generation process.

To facilitate LLM numerical reasoning, historical and forecast data are formatted as key-value pairs, each entry including a date and its corresponding average price (e.g., “[[August 2022, 10,890], [December 2022, 11,585]]”). If historical records for specific months are unavailable, these months are simply omitted from the provided data.

Despite its effectiveness in handling forecast-based reasoning queries, LLMs often provide verbose explanations rather than succinct numerical predictions for straightforward forecasting questions Chen et al. (2025b). To mitigate this, we introduce an LLM-driven numerical extraction module, which employs in-context learning with five examples, explicitly prompting the model to return only the essential numerical results. All detailed prompts utilized within the analysis agent are comprehensively documented in Appendix E.

ODTQA-FoRe consists of two subtasks: time-series forecasting and forecast-based reasoning. We evaluate time-series forecasting using Mean Squared Error (MSE), Mean Absolute Error (MAE), and Mean Relative Error (MRE). For forecast-based reasoning, we use Accuracy, Precision (P), Recall (R), and F1-score (F1). To account for questions with multiple items, we compute F1 at the item level for partial credit, whereas Accuracy requires an exact match of all items within a single question.

For the core time-series forecasting task, we evaluate eight models to select the optimal forecaster for our TimeFore framework. These include four lightweight models (TimesNet, TimeMixer, TimeXer Wang et al. (2024b), WPMixer Murad et al. (2025)), two LLM-based time-series models (AutoTimes Liu et al. (2024b), Time-MoE Shi et al. (2025)), and two general-purpose LLMs (Qwen3 30B, GLM 4.5 Air). The general-purpose LLMs perform forecasting via in-context learning (ICL) on the original data, while the six specialized models are trained on data imputed by TimesNet.

As Table 2 shows, the specialized time-series models significantly outperform the general-purpose LLMs, which confirms the latter’s relative weakness in direct prediction tasks. Among all candidates, TimeXer achieves the best performance and is therefore selected as the forecasting component within the TimeFore framework.

Further details on experimental setups—including retrieval agent performance, training data specifics, and other implementation details—are provided in Appendix C due to space constraints.

As ODTQA-FoRe is a newly proposed benchmark without directly comparable baselines, we introduce a Vanilla baseline, which first uses the BM25 algorithm to retrieve relevant table captions, then constructs a five-shot prompt for in-context learning (ICL), and finally relies entirely on the LLM’s intrinsic capabilities to predict future data.

As presented in Table 3, the experimental results demonstrate that our TimeFore framework significantly outperforms the vanilla baseline across all models. This outcome highlights the inherent limitations of general-purpose LLMs when applied directly to specialized time-series forecasting tasks.

These results collectively demonstrate that delegating forecasting to a specialized time-series model, rather than relying on the LLM itself, not only enhances prediction accuracy but also significantly improves performance on subsequent reasoning tasks. This finding offers a key insight for the future development of LLM-based predictive and reasoning systems: a hybrid approach that leverages specialized tools for their respective strengths yields a more robust and accurate overall framework.

To identify performance bottlenecks and quantify the contribution of each component within our multi-step TimeFore pipeline, we conduct a comprehensive ablation study. For the forecast-based reasoning task, we systematically substitute individual model-generated outputs with their ground-truth counterparts and observe the resulting performance gains. This allows us to isolate the impact of table retrieval, data retrieval (SQL generation), and time-series forecasting.

The study is designed around four specific configurations. The first, denoted as GH (NF) for Golden History (No Future), serves as a baseline where the LLM receives only ground-truth historical data and no future data. This measures the model’s reasoning ability without any forecasting input. The second configuration, +GTC, provides ground-truth table captions to the Retriever to isolate the impact of the initial table retrieval step, while the rest of the pipeline remains unchanged. The third setup, GH + PF (Golden History + Predicted Future), bypasses the Retriever by providing ground-truth historical data directly but still requires the Forecaster to predict future values, thereby isolating the performance impact of the SQL generation stage. The final configuration, GH + GF (Golden History + Golden Future), establishes the theoretical upper bound of the LLM’s reasoning capability by providing the Analyzer with perfect historical and future data, bypassing both the Retriever and the Forecaster.

The results of this study, visualized in Figure 3, reveal a clear performance hierarchy. The most significant performance drop occurs in the GH (NF) setting, confirming that reasoning without forecasting is largely ineffective. While providing golden table captions (+GTC) or golden historical data (GH + PF) yields only marginal improvements, the most substantial performance gain by far is achieved in the GH + GF setting when perfect future data is supplied.

This unequivocally demonstrates that the accuracy of the time-series forecasting component is the primary bottleneck for the entire framework. Accurate numerical prediction is not just beneficial; it is a prerequisite for enabling the large model to perform reliable downstream reasoning.

More detailed ablation study for Analyzer’s numerical extraction module are available in Appendix C.5.