Verifying DLMF with Maple and Mathematica

This wiki supports an anonymous submission to CIKM 2021[1]. Since the CIKM conference uses a double-blind review system, the identity of the authors is hidden. For legal inquiries, use the methods described at https://wikitech.wikimedia.org/.

In the following we present the results of our translations and evaluations.

Bug Reports

You can find a PDF with commands that illustrate the encountered errors in Mathematica here: File:Mathematica Bugs Overview.pdf

We provide the same file in the Wolfram system notebook format (NB) here: File:Mathematica Bugs Notebook File.nb

DLMF Translations and Results

This section explains the data you can access. Below, you will find a large table that shows the overall results of translations, symbolic and numeric verifications, as well as a number of errors and aborted calculations for each DLMF chapter. Before you move on to investigating the results, please have a look at how to read the data first.

How to read the data?

The data on this website can be considered as an extension of the DLMF but not a copy. Some labeled equations in the DLMF do not appear in this data because they were skipped or caused translation errors in both systems (Maple and Mathematica). In addition, the result tables on this website do not provide context! For checking the context, every equation in our dataset is linked directly to the equation in the DLMF. You can access the original DLMF equation by clicking on the label in the most-left column in the result tables.

You can access the results by clicking on a particular chapter below. This will load a page showing links to each section. Red links indicate that this section is empty, i.e., no results are available for that section. Clicking on a blue inked link will load the result table for that section. Below you see an example of such a table with just two equations from different chapters. As you can see, the first entry 4.12.E1 is grayed out. This happens when the entry was skipped due to missing semantic macros in the source of the test case (flagged as Skipped - no semantic math). You can see the actual semantic LaTeX source of every test case below the equation. Since most of these cases are not translated properly by LCT, we skipped them directly, i.e., they were not further processed for symbolic or numeric evaluations. In fact, those cases have not even been translated into our pipeline. However, we added the translations just for our result tables because they may provide further interesting insights into the translation process.

The most-right columns in the table show the numeric evaluation results. If an equation was not numerically verified, it shows Failed [x/y] where Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle x} is the number of failed calculations and Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle y} is the number of all test calculations for this test case. As you can see in the second entry below 11.5.E2, 15 test calculations failed out of 25, so 10 were successful. When you click on the little Expand, we show the first couple of results for the failed test cases. Since the tables are already very large, we cannot show all failed test calculations.

Clicking on expand for the second entry in our example table below, we see the test result and the test values that produced this result. As explained in the paper, the calculation was simply performed on the difference of the left- and right-hand side of the equation LHS - RHS. Further explanations on why this test failed can be found in the paper.

Failed [15 / 25]

Result: Complex[0.9495382353861556, -0.46093572348323536]
Test Values: {Rule[z, Power[E, Times[Complex[0, Rational[1, 6]], Pi]]], Rule[ν, 1.5]}

Result: Complex[0.7706973036767981, -0.20650772012904173]
Test Values: {Rule[z, Power[E, Times[Complex[0, Rational[1, 6]], Pi]]], Rule[ν, 0.5]}

... skip entries to safe data

DLMF	Formula	Constraints	Maple	Mathematica	Symbolic Maple	Symbolic Mathematica	Numeric Maple	Numeric Mathematica
4.12.E1	Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \phi(x+1) = e^{\phi(x)}} \phi(x+1) = e^{\phi(x)}	Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle -1 < x, x < \infty}	phi(x + 1) = exp(phi(x))	\[Phi][x + 1] == Exp[\[Phi][x]]	Skipped - no semantic math	Skipped - no semantic math	-	-
11.5.E2	Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \StruveK{\nu}@{z} = \frac{2(\tfrac{1}{2}z)^{\nu}}{\sqrt{\pi}\EulerGamma@{\nu+\tfrac{1}{2}}}\int_{0}^{\infty}e^{-zt}(1+t^{2})^{\nu-\frac{1}{2}}\diff{t}} \StruveK{\nu}@{z} = \frac{2(\tfrac{1}{2}z)^{\nu}}{\sqrt{\pi}\EulerGamma@{\nu+\tfrac{1}{2}}}\int_{0}^{\infty}e^{-zt}(1+t^{2})^{\nu-\frac{1}{2}}\diff{t}	Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \realpart@@{z} > 0, \realpart@@{(\nu+\tfrac{1}{2})} > 0, \realpart@@{(\nu+k+1)} > 0, \realpart@@{((-\nu)+k+1)} > 0, \realpart@@{(n+\nu+\tfrac{3}{2})} > 0}	StruveH(nu, z) - BesselY(nu, z) = (2*((1)/(2)*z)^(nu))/(sqrt(Pi)*GAMMA(nu +(1)/(2)))*int(exp(- z*t)*(1 + (t)^(2))^(nu -(1)/(2)), t = 0..infinity)	StruveH[\[Nu], z] - BesselY[\[Nu], z] == Divide[2*(Divide[1,2]*z)^\[Nu],Sqrt[Pi]*Gamma[\[Nu]+Divide[1,2]]]*Integrate[Exp[- z*t]*(1 + (t)^(2))^(\[Nu]-Divide[1,2]), {t, 0, Infinity}, GenerateConditions->None]	Successful	Successful	-	Failed [15 / 25] Result: Complex[0.9495382353861556, -0.46093572348323536] Test Values: {Rule[z, Power[E, Times[Complex[0, Rational[1, 6]], Pi]]], Rule[ν, 1.5]} Result: Complex[0.7706973036767981, -0.20650772012904173] Test Values: {Rule[z, Power[E, Times[Complex[0, Rational[1, 6]], Pi]]], Rule[ν, 0.5]} ... skip entries to safe data

As a summary, here is an explanation for every column of the result tables.

DLMF

Contains the equation label as a link to the actual equation in the DLMF. Click on the label if you need to check the context of a formula in the DLMF.

Formula

Contains the rendered formula and the actual semantic LaTeX source (sometimes also referred to as content-tex). More information about the semantic (or content) LaTeX can be found in

Constraints

This shows the constraints that LCT has generated by analyzing the definitions of the functions that are included in the equation. This also includes the constraints that are directly attached to a test case. For example, 11.5.E2 only has the constraint Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \realpart@@{z} > 0} . LCT followed the definitions of the involved functions and identified several more constraints for this test case. A detailed explanation of this process was given in the paper. Note, however, that this list of constraints does not contain the manually defined global constraints that we applied for all test cases (as shown in Table 3 in the paper). This way, the result tables only show what LCT generated and applied for the numeric evaluations.

Maple

The translation to Maple

Mathematica

The translation to Mathematica

Symbolic (Maple / Mathematica)

The result of the symbolic evaluation. If this was skipped, no calculations have been performed on the test case.

Numeric (Maple / Mathematica)

The result of the numeric evaluation as explained above and in the paper.

Links to the data

In the following, we present an overview of the translations of the DLMF equations to the CAS Maple and Mathematica.

DLMF	Formula	Translations Maple	Translations Mathematica	Symbolic Evaluation Maple	Symbolic Evaluation Mathematica	Numeric Evaluation Maple	Numeric Evaluation Mathematica
DLMF	6,545	4,114 (62.9%)	4,713 (72.0%)	1,084 (26.3%)	1,235 (26.2%)	698 (26.7%)	784 (22.6%)

The following links provide access to all results for each DLMF chapter. This will show the sections in each chapter. Please notice that many sections in a chapter might be empty (indicated by a red link color instead of a blue link). This happens when all formulae in a section has been skipped or the case analyzer was not able to parse the expression properly. In some cases, the section simply did not contain any formula.

1. Algebraic and Analytic Methods
2. Asymptotic Approximations
3. Numerical Methods
4. Elementary Functions
5. Gamma Function
6. Exponential, Logarithmic, Sine, and Cosine Integrals
7. Error Functions, Dawson’s and Fresnel Integrals
8. Incomplete Gamma and Related Functions
9. Airy and Related Functions
10. Bessel Functions
11. Struve and Related Functions
12. Parabolic Cylinder Functions
13. Confluent Hypergeometric Functions
14. Legendre and Related Functions
15. Hypergeometric Function
16. Generalized Hypergeometric Functions and Meijer G-Function
17. q-Hypergeometric and Related Functions
18. Orthogonal Polynomials
19. Elliptic Integrals
20. Theta Functions
21. Multidimensional Theta Functions
22. Jacobian Elliptic Functions
23. Weierstrass Elliptic and Modular Functions
24. Bernoulli and Euler Polynomials
25. Zeta and Related Functions
26. Combinatorial Analysis
27. Functions of Number Theory
28. Mathieu Functions and Hill’s Equation
29. Lamé Functions
30. Spheroidal Wave Functions
31. Heun Functions
32. Painlevé Transcendents
33. Coulomb Functions
34. 3j,6j,9j Symbols
35. Functions of Matrix Argument
36. Integrals with Coalescing Saddles

Examples

A good starting point might be Combinatorial Analysis - 26.3 Lattice Paths: Binomial Coefficients. All expressions are relatively straight-forward. Nonetheless, 23.3.E8 failed numerically

Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \binom{m}{n} = \sum_{k=0}^{n}\binom{m-n-1+k}{k}}

This test case failed because Mathematica simplified the right-hand side directly to Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \frac{(1+n) {m \choose n+1}}{m-n}} which would result in a division by zero for Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle m = n} . Indeed, our result table show that the test evaluation failed for Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle m=n=1} and Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle m=n=2} (the last case Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle m=n=3} was omitted because we only show the first two failed calculations to reduce the size of the result tables). This simple example shows the challenge of interpreting the data and working with computer algebra systems.

Translations and Evaluations Overview Table

	Meaning
2C	Chapter Code
S	Successful
%	Percentage
F	Fail
P/T	Partially / Totally Failed
A	Aborted
E	Errors
Base	The baseline performance of the translator
Maple	The CAS Maple 2020
Mathematica	The CAS Mathematica

					Symbolic						Numeric
Formulae		Translations			Maple			Mathematica			Maple						Mathematica
2C	Total	Base	Maple	Math	S	%	F	S	%	F	S	%	F	[P/T]	A	E	S	%	F	[P/T]	A	E
1. AL	227	60	102	103	36	35.3%	60	34	33.0%	69	14	23.3%	35	[ 12 / 23]	7	4	14	20.3%	40	[ 9 / 31]	11	4
2. AS	136	33	65	65	6	9.2%	47	6	9.2%	59	7	14.9%	33	[ 5 / 28]	1	5	4	6.8%	38	[ 6 / 32]	7	9
3. NM	53	36	40	40	6	15.0%	31	5	12.5%	35	1	3.2%	27	[ 9 / 18]	0	2	0	0.0%	29	[ 8 / 21]	6	0
4. EF	569	353	494	564	270	54.7%	221	304	53.9%	260	88	39.8%	126	[ 64 / 62]	0	6	110	42.3%	146	[ 55 / 91]	2	0
5. GA	144	38	130	139	41	31.5%	76	65	46.8%	74	39	51.3%	25	[ 12 / 13]	4	8	30	40.5%	20	[ 9 / 11]	13	9
6. EX	107	21	56	77	13	23.2%	43	18	23.4%	59	10	23.2%	31	[ 13 / 18]	0	2	23	39.0%	32	[ 6 / 26]	4	0
7. ER	149	35	101	120	52	51.5%	47	45	37.5%	75	21	44.7%	23	[ 10 / 13]	2	1	21	28.0%	43	[ 13 / 30]	9	1
8. IG	204	84	160	163	51	31.9%	102	65	39.9%	98	27	26.5%	61	[ 20 / 41]	9	5	22	22.4%	44	[ 19 / 25]	16	15
9. AI	235	36	180	179	54	30.0%	124	69	38.5%	110	34	27.4%	75	[ 41 / 34]	4	8	30	27.3%	58	[ 38 / 20]	14	7
10. BS	653	143	392	486	80	20.4%	209	115	23.7%	371	86	41.1%	59	[ 41 / 18]	52	12	90	24.2%	151	[ 57 / 94]	92	18
11. ST	124	48	121	112	39	32.2%	73	36	32.1%	76	25	34.2%	40	[ 14 / 26]	3	5	21	27.6%	33	[ 8 / 25]	10	11
12. PC	106	33	79	90	25	31.6%	50	18	20.0%	72	15	30.0%	24	[ 15 / 9]	11	0	13	18.0%	43	[ 15 / 28]	12	3
13. CH	260	126	252	254	75	29.8%	143	69	27.2%	185	14	9.8%	90	[ 55 / 35]	37	2	23	12.4%	95	[ 59 / 36]	45	21
14. LE	238	166	230	229	30	13.0%	163	30	13.1%	199	40	24.5%	93	[ 57 / 36]	18	12	59	29.6%	92	[ 54 / 38]	41	5
15. HY	206	148	198	197	46	23.2%	115	53	26.9%	144	17	14.8%	52	[ 34 / 18]	23	23	23	16.0%	77	[ 52 / 25]	29	6
16. GH	53	20	23	25	3	13.0%	16	2	8.0%	23	1	6.2%	9	[ 8 / 1]	6	0	1	4.3%	10	[ 7 / 3]	9	2
17. QH	175	1	53	124	23	43.4%	24	6	4.8%	118	0	0.0%	0	[ 0 / 0]	1	23	13	11.0%	57	[ 52 / 5]	39	5
18. OP	468	132	235	288	65	27.6%	148	101	35.1%	185	67	45.3%	50	[ 32 / 18]	14	17	45	24.3%	68	[ 31 / 37]	52	12
19. EL	516	103	252	416	39	15.5%	192	51	12.2%	365	18	9.4%	123	[ 44 / 79]	34	17	18	4.9%	264	[ 49 / 215]	61	15
20. TH	128	52	98	98	10	10.2%	68	1	1.0%	97	0	0.0%	32	[ 17 / 15]	20	16	33	34.0%	40	[ 25 / 15]	24	0
21. MT	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
22. JA	264	115	232	238	46	19.8%	176	30	12.6%	206	20	11.4%	116	[ 25 / 91]	36	4	22	10.7%	131	[ 39 / 92]	51	0
23. WE	164	7	19	34	1	5.3%	16	4	11.8%	30	0	0.0%	14	[ 2 / 12]	1	1	2	6.7%	23	[ 9 / 14]	2	3
24. BP	175	31	117	148	15	12.8%	101	23	15.5%	125	67	66.3%	32	[ 19 / 13]	1	1	78	62.4%	33	[ 22 / 11]	14	0
25. ZE	154	28	124	120	19	15.3%	90	48	40.0%	72	43	47.8%	29	[ 18 / 11]	10	8	22	30.5%	22	[ 6 / 16]	22	3
26. CM	136	31	78	87	20	25.6%	50	19	21.8%	68	30	60.0%	11	[ 8 / 3]	2	7	44	64.7%	18	[ 10 / 8]	5	1
27. NT	79	5	26	15	3	11.5%	17	6	40.0%	9	2	11.8%	6	[ 3 / 3]	0	8	3	33.3%	6	[ 3 / 3]	0	0
28. MA	267	52	97	110	7	7.2%	80	7	6.4%	103	6	7.5%	32	[ 12 / 20]	26	15	3	2.9%	48	[ 13 / 35]	33	17
29. LA	111	11	23	22	0	0.0%	21	0	0.0%	22	0	0.0%	19	[ 2 / 17]	0	2	0	0.0%	21	[ 1 / 20]	0	1
30. SW	71	14	19	26	0	0.0%	18	0	0.0%	26	0	0.0%	18	[ 5 / 13]	0	0	0	0.0%	19	[ 2 / 17]	5	1
31. HE	35	29	22	15	5	22.7%	13	2	13.3%	13	2	15.4%	10	[ 0 / 10]	0	1	0	0.0%	8	[ 0 / 8]	5	0
32. PT	67	43	57	57	3	5.3%	51	3	5.3%	54	1	2.0%	44	[ 7 / 37]	4	2	0	0.0%	41	[ 2 / 39]	8	5
33. CW	108	21	14	11	1	7.1%	13	0	0.0%	11	0	0.0%	5	[ 2 / 3]	0	8	0	0.0%	11	[ 2 / 9]	0	0
34. TJ	57	0	1	37	0	0.0%	1	0	0.0%	37	0	0.0%	1	[ 0 / 1]	0	0	14	37.8%	10	[ 5 / 5]	13	0
35. FM	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
36. IC	106	12	24	24	0	0.0%	19	0	0.0%	24	3	15.8%	12	[ 1 / 11]	3	1	3	12.5%	13	[ 1 / 12]	1	6
Failed to parse (LaTeXML (experimental; uses MathML): Invalid response ("") from server "http://latexml:8080/convert/":): {\displaystyle \sum}	6545	2067	4114	4713	1084	26.3%	2618	1235	26.2%	3474	698	26.7%	1357	[607 / 750]	329	226	784	22.6%	1784	[687 / 1097]	655	180